Computer engineering magic

"Any sufficiently advanced technology is indistinguishable from magic." Arthur C. Clarke

In the field of computer engineering, the following appear as magic to me. I am planning to lift this mystery by building small-scale models:

1. Detecting syntax errors while typing (linting)
2. Code completion
3. Deserializing JSON into a class instance
4. Stopping a program at a breakpoint and inspecting debug information
5. Interpreters (with parallelism on multicore) and compilers
6. How games like Prince of Persia could be written in assembly

These would help me to transition from "user of tools" to "creator of systems."

Kanban vs. Scrum

Kanban is highly flexible with no fixed timeframes. In Scrum, work effort has to be estimated and organized into fixed-length iterations, known as sprints, typically lasting 2-4 weeks.

Kanban is better for projects with changing priorities, while Scrum is more suited for projects with defined requirements and deliverables. If continuous delivery is essential, Kanban might be the better choice. For projects benefiting from iterative development, Scrum is preferable.

For the type of work I am doing, I prefer using the Kanban method, incorporating daily stand-up meetings where each person speaks for no more than 5 minutes. This ensures that everyone stays focused and that efforts are concentrated on high-priority issues.

Outsourcing software development

When outsourcing parts of a software development project, you'll need to overcome several challenges:

Clearly Define the Problem and Verification: It's essential to have a detailed design and create prototypes for testing your ideas before writing a statement of work. The outsourcing partner mainly contributes in terms of usability and maintainability, i.e. they should convert the proof of concept that can barely be used by experts to a product that can easily be used by normal people.

Identify Reliable Partners: This is arguably the most challenging step. In the software world, many claim expertise – after all, it's just coding, right? However, the ideal scenario is knowing firsthand that the team or individuals you're considering are truly capable of handling the project. If you are not sure, starting with small scale (~1 week) discovery projects (free of charge) will be helpful.

Manage Budgetary Constraints: As costs rise, so does the bureaucracy and the number of required approvals. One effective way to control cost is to narrow the project's scope and build on top of open source. While it's imperative to obtain the source code from your partner, consider handling documentation, formal verification and maintenance internally.

By adhering to these guidelines, you maintain control over the project and its future maintenance. You also gain access to top-tier talent that might not otherwise join your team, all while potentially reducing costs compared to in-house development.

Filtering software product ideas

A typical response I get from people when they learn that I am a software developer is, "I have this app idea X, does it have potential? Would you be willing to implement it?" In software, any idea can have potential, so dismissing it outright is not logical. My default answer is, "That's a fine idea. If you write a concept of operation document detailing how the app will be used and why it would be useful, I promise that I will design and code your first version (MVP) for free," and I genuinely mean it.

This approach benefits me in three ways. Firstly, by refraining from saying 'no', I show respect for their ideas and creativity, thereby avoiding potential hurt feelings. Secondly, it allows me to focus on truly promising and well-thought-out projects, rather than getting bogged down in evaluating every idea that comes my way. Lastly, if an idea does prove to be good, I gain the opportunity to create value.

However, despite my promise to work for free, I rarely receive a concept of operation document. It's easy to have an idea but difficult to write it down. Also, when writing it down, they think more about it and usually realize that it wasn't as impressive as they initially thought.

Music: Damon Albarn - Sister Rust

Becoming a 100X Engineer

To become a 100X engineer, always ponder the following questions before attempting to solve a problem:

1. Is this problem worth solving right now?
2. Would buying an existing solution be feasible?
3. Would outsourcing be feasible?
4. What is the simplest design to solve this problem? Simple is smooth and smooth is fast.

Music: The Wellerman (Sea Shanty) 

Fantasy and quality

Estimating how long a complex project will take is difficult, especially if it contains software development. Experienced engineers can estimate cost based on past projects. However, I am puzzled to see time and time again senior managers who come up with overly optimistic schedules. 

One reason might be to motivate employees to work harder. The novices will think that a short duration overtime will solve the crisis and they will be back to normal where in fact it will take months. You can fool people only once, and when they find out, you won't be able to motivate them again.

Another reason is to avoid conflict with upper management and after some time, starting to believe in your own optimisim, see Waltzing with Bears.

When management continues to live in fantasy land, after a couple of months people will be disillusioned, morale drops and the project takes longer with lower quality than would be the case at a normal pace. This type of mismanagement is only rational if the manager is expecting a change of position soon, meaning that he/she won't have to deal with the mess. 

Quality suffers because everyone thinks that this will be over quickly with a firm push, resorting to quick and dirty fixes, reasoning that the mess will be cleaned up soon. The longer the crisis mode drags on, the more technical debt is accumulated, which in the end becomes so large that no one is eager to refactor the system, fearing that the slightest change will break it. This leads to high cost maintenance and can cause catastrophic failures whose root cause can be impossible to find. Due to endless rework and troubleshooting, total cost of project increases dramatically.

Intellectual work like software development requires good concentration which in turn requires a rested, motivated individual. Long overtimes are like killing the goose that laid golden eggs.

My suggestions in a nutshell:

1. If you are a manager, avoid more than a couple of weeks per year overtime. Have a look at my reading list. Always look at past data, be aware that avoiding conflict with upper management might lower your stress in the short run but it has a high price down the road.
2. If you are one of the "managed" ones, politely point out the risks. But if you see that no one is interested in that, shut up and concentrate on improving the quality of your work and documentation so that when disaster strikes, you will be able to quickly demonstrate that your piece of work is not to blame. Read Death March.

Software product maturity levels

Software products have three maturity levels:

1. Product sometimes works.
2. Product usually works.
3. Product always works.

Transition from lower to higher maturity levels is achieved through experienced personnel, better development practices and tests.

JavaScript: Adding to arrays with "..."

In JavaScript, push method is used to add to an array. Let's say you have an existing array called out  = [{e1: 5}, {e2:10}] and want to add its elements to an array called arr. Results differ for arr.push(out) and arr.push(...out). When you use ellipsis (...), each element of out array will be added as a separate element to arr and arr.length will increase by 2 after push. Without ..., out will be added as a single element to arr, and arr.length will increase by 1 after push operation. Below is an example script:

Technical Team Leader

A software technical team leader's main responsibility is to make the team better, because that will result in highest efficiency due to each member becoming a better version of themselves. Examples of duties of a techlead:

• Review/refactor design and documentation that the team produces. See Scrum sprints.
• Review code, optimize/refactor for comprehensibility, speed, memory and safety. This will keep you up to date with the code.
• Evaluate and select code analysis tools.
• Evaluate tutorials and point team members to them.
• Use stack instead of heap.
• Solve difficult technical challenges that are beyond the capabilities of a team member.
• Assist in solving difficult problems. Since you have been continuously reviewing design and code, you won't need extra acclimation time.
• Review issue definitions and refactor for comprehensibility.
• Review commits to repository (commits should be multiple times a day, not once a week)
• Write simulators/emulators for external hardware and software components
• Write initial versions of build scripts. Scripts will be maintained by juniors.
• Write initial versions of setup scripts that makes setting a development environment as simple as a single mouse click.  Scripts will be maintained by juniors.
• Find ways to decrease build times, create tutorials.
• Disable antivirus
• Use Linux instead of Windows.
• Precompiled headers
• Optimize IDE settings, create tutorials.
• Shortcuts
• Folder view
• Output directory = bin

Notice that the list does not contain product work because fulfilling the above would take a lot of time. Product development consists mostly of repetitive/boring work that a junior can handle with guidance of a senior. By freeing the techlead from those chores and making him a team mentor and solver of only hard problems, you improve productivity of everyone.

Writing my first C program in Linux

Recently I wrote my first C program in Ubuntu Linux for which I had a deadline. I was already familiar with C/C++ on Windows. It was quite a learning experience. My method was as follows:

1. Search the internet for projects matching the problem definition. Result: There weren't any.
2. Search examples/tutorials for terms in the problem definition like fork, poll.
3. Write small demos. Modify and combine examples to solve the original problem. Result: Faced many difficulties during adaptation.
4. Dive deeper and understand the details of concepts/terms that I am not well versed in, like pipes.
5. Apply that deeper knowledge.
6. Increase depth until the problem is solved in its entirety.

As you can see, I did not start by reading a lengthy Linux C fundamentals book. I focused on the problem at hand, tried shortcuts first and only spent more time on problematic points. I felt a lot of frustration while trying to hack my way out but I was determined enough to overcome constant failures. If you are working under time pressure, this is the fastest method of getting things done.

Notes:

• To display processes starting with "node": ps -ef|grep node
• To run as a background process, add & to the end.
• Windows Subsystem for Linux, using Ubuntu with VS Code, tasks.json to build file (ctrl + shift+b)
• pipes, socketpair, filedescriptor, dup2. 
• Redirecting stdin
• fork - exec
• poll
• Converting a string to a vector of strings, with space as delimiter
• Difficulty of using strings. Writing and reading structures is much easier, to read a string after a write, you need to send EOF with close(fd), but then you have the problem of opening the filedescriptor again.
• gcc, makefile, tar
• Difficult to find bug in C: for (int i = 0; len; i++). Note that i< is missing.
• When you print elements of a union structure, you will only get the last set element right.

Remote work challenges

The global coronavirus pandemic has forced many institutions to think seriously about remote work. The first idea that comes to mind is make people work on laptops. If the work place had to use a local network isolated from the internet due to security concerns, the following challenges have to be faced:

• A lot of laptops have to be reconfigured by the IT department so that they cannot be connected to the internet. Software needed for daily work have to be installed. That's a lot, depending on the number of laptops and the number of software to be installed on each laptop.
• Many software products needed for development rely on local license servers. How are you going to make them work on laptops that are neither connected to your LAN nor the internet? You may have to obtain standalone licenses (extra cost) and install them on each laptop. In my case, I need licenses for two different products. May be existing license servers can be made to work on a virtual machine on the laptop.
• Complex projects require multiple people to work on the same software components. How will you prevent merge conflicts when the oubreak has calmed down and everybody wants to push their work to the local repository?
• Embedded software projects require multiple people and hardware working together. You might take your hardware home but you won't be able to get your colleagues and their hardware/software due to social isolation requirements to prevent the virus spread.
• If you have kids at home, how will you prevent them from constantly interrupting you? By having a schedule and clear visual indicators that you are working (like a sign on the door).

If your software/hardware were highly modular, if you had abstractions/simulators, you could take only the parts that are suitable to work at home. But true modularity is almost never the case. In my opinion, during the remote period it is not realistic to expect work to be done as usual. The main areas to focus are:

• Limited development with only your hardware and software (if standalone licenses can be purchased).
• Improving documentation of existing software/design. Documenting the system will show you clearly where the design is too complex and needs improvement.
• Self education. Example subjects for me: Blockchain, real time operating systems, embedded software development.
• Daily physical exercise.

02.08.2022: TopTal The Suddenly Remote Playbook

Letters to a novice programmer

Dear novice, for you to change the way you develop software, you have to go through the following phases:

1. A lot of work you thought would take 1 day ends up taking 1 month because most of the code you reuse is leaky, which causes unforeseen extra work.
2. You get annoyed that your plans and deadlines are actually worthless with this type of low quality code base. You get so irritated that you want to do something about it.
3. You start to make a list about possible refactorings that would increase the robustness of existing code. You add items to this list whenever you see something fishy in code, but you can't act on it right away because you almost certainly have  some crisis that needs fixing immedately.
4. Whenever there is a break from firefighting mode, you go back to your list and start working on the most important items.
5. You realize that most of your refactorings will have unforeseen side effects and that you have to revert them. You give up refactoring at this point and go back to firefighting. 
6. After wasting 10 years, you realize that you have to refactor on the off chance of improving quality to the point where you can reuse code in peace. And you are enlightened (!)

A novice programmer was once assigned to code a simple financial package. The novice worked furiously for many days, but when his Master reviewed his program, he discovered it contained a screen editor, a set of generalized graphics routines, and an artificial intelligence interface, but not the slightest hint of anything financial. When the Master asked about this, the novice became indignant. "Don't be so impatient," he said, "I'll put in the financial stuff eventually."

Notice that I have not mentioned any software development standards or processes. Their time comes after you have matured through these phases. The most important one is phase 2, i.e. to be annoyed of wasting time by not being able to trust existing code, ending with rewrites whenever you "reuse" code.

Music: Kraftwerk - Das Model

Effect of good software design on maintenance and innovation

If the design of a complex software project is poor, a lot of effort has to be spent to maintain the system. If design is good, more time will be freed up for innovation instead of wasting time to keep the product barely working.

Music: The Stable Song - Gregory Alan Isakov (Peanut Butter Falcon soundtrack)

Solving inheritance ambiguity with composition

Let's say you have the following C++ classes, making use of the observer pattern:

In implementation of class A, you call Notify(T) of CObservable.

When you compile, you will get a message that there is ambiguity in calling Notify(), because the compiler does not know if it should call Notify(T) with class T1 as a parameter or T2.

The solution is to remove CObservable from inherited classes and introduce a field, i.e. using composition over inheritance. You also need to have a registerT2Observer() method that in turn calls the register method of CObservable:

Letters to a novice programmer

Dear novice, this time it has been a short while, isn't it? When you have a dll that you call from both a Java app and C++ app with the same inputs but the outputs differ, do not suspect that there might be something wrong with the operating system or JNI. Your problems will always be due to something you did wrong.

You should write some debug code inside the C++ dll that prints dll inputs to a file. Run the dll from Java and C++ and compare dll inputs. I guarantee you that you will see a difference there because most probably you did something wrong before passing the inputs to dll like using a wrong index or making an erroneous unit conversion.

Another tip: When comparing files, never trust your eyes, always use a comparison tool like Total Commander's "Compare by content" or Beyond Compare.

Letters to a novice programmer

Dear novice, it has been a while since my last letter. Today I want to stress the importance of revisiting your code every couple of months or so and asking yourself "could I achieve the same functionality with less code?". In other words, be your own garbage collector. If you do this a couple of times, your code might graduate from being miserable to being tolerable. And who knows, one day it might even attain the coveted status of being "elegant"?

Today's example concerns avoiding unnecessary local variables to reduce clutter:

Music: Bomfunk MC's - Freestyler (OQ Trap Remix)

The myth of developer independent systems

Today a colleague of mine was asserting that the core reason of sloppy development is not having detailed quality procedures, i.e. bureaucratic systems independent of who is following them. I told him that we have more than enough development procedures, that any complex system is always dependent on the quality of people who implement them. If a project is sufficiently complicated and there is time pressure and market competition, you need high quality people who can evaluate risks realistically, tailor standards and cut corners to save time and effort. But they are also acutely aware of technical debt and clean up the quick and dirty solutions when time permits it.

You never have enough resources to develop your ideal system and always have to settle on a viable product. If your chief engineers are not wise enough, they will underestimate complexity, confuse priorities and take unreasonable risks. The problem is that they won't know that those risks are high. For them it will look like a brilliant course of action, until disaster strikes. If they are bad enough, they won't even be able to learn the correct lessons from failure.

Bad products are not the result of bad procedures but bad engineers and managers. The shortest path to success is to find and nurture good people who can develop quality products, write realistic procedures and make the team better.

Letters to a novice programmer: Content priority

It has been a long time since our last letter. Today I would like to talk about prioritizing content. When writing code or documenting a complex (more than 1 man*month of effort) project, you should always start with high priority features/content first. For example, if you are writing a developer handbook for a piece software, start with the most frequent use case first. Do not write page after page about design details that are not important for that particular use case. You will NEVER have enough time to write about everything you think is necessary. By documenting/coding high priority items, you maximize document/software usefulness.

Get SVN revision and write to header file

I am creating a C++ dll file from a MATLAB Simulink model. The simulink model repository is hosted on subversion. I needed a way to embed simulink model subversion revision to dll so that the dll could return that revision via an interface method to interested clients. I wrote the following batch file to fetch revision from SVN and write to a header file:

Why I prefer exceptions to error codes

Why I prefer exceptions to error codes:

Error codes are far safer for well-reviewed, critical code ...for most code, failing early [with exceptions] is better simply because it always makes debugging easier – even if it doesn't make the impact of the error smaller... With an exception, you get a call stack, and an error string from the bottom layer.