Saturday, December 30, 2017

Practical application of the dot product

I recently had to write an algorithm to decide whether two trains (whose movement directions are known) are moving towards each other or not.

When two trains move towards each other the angle between their direction vectors is larger than 90 degrees. If the track was a straight line the angle would be close to 180 degrees, but tracks are not always straight.

I used the dot (scalar) product as follows:


  1. Calculate the dot product of the train direction vectors. This is equal to cos(angle).
  2. If 90 < angle <= 180 (-1 =< cos(angle) < 0), the two trains are moving towards each other
Note 1: If the track is bending more than 90 degrees, this algorithm will not work:
Note 2: If the trains have passed each other just using the dot product won't work, you also have to check whether the trains are in front of each other (let that be another blog post):

Friday, December 22, 2017

How to respond to questions of novice engineers

When a novice engineer comes to you with a technical question, your first reponse should not be answering the question at hand but asking "what are you trying to accomplish" because most of the time the novice will have gaps in his understanding of the job to be done. Based on that misunderstanding, he will try to solve some technical problem, get stuck and come to you for guidance.

By clarifying the higher level goal to be reached, you can be sure that the question is indeed meaningful and answering it will be productive.

Further reading: Difference between novice and experienced engineers

Music: Hallelujah - Pentatonix

Friday, December 15, 2017

Keeping motivation after lots of failures

Software development means getting stuck a lot of times where all your attempts at moving forward result in failure. This affects my motivation and after enough failures, I feel desperate and don't want to touch the project anymore.

One way to overcome this deadlock is to do smaller things with guaranteed success. For example, I improve documentation, draw prettier graphs, make more detailed plans. Scoring little victories shortens my procastination period and I gain enough enthusiasm to tackle the beast.

The 7 Habits of Highly Effective Artists

Monday, November 27, 2017

Mobile Games for Kids

Below is a list of mobile (Android) games that my 5 year old plays on his tablet and I recommend:
  • Minecraft
  • Monument Valley 2
  • Alto's Adventure
  • Samorost 3
  • Cat Quest
  • War Robots
  • Badland 2
  • Contre Jour
  • World of Goo
  • Bowmasters

Thursday, October 19, 2017

Stop worrying that your expertise isn't worth much

Today I was talking with a chemist friend of mine who complained that he could rarely use his chemical expertise and was spending most of his time with jobs at the technician level. While I am in a slightly better situation, mine is not that much different.

My expertise is in dynamic flight simulations of 6DoF rigid bodies. Outside of my day job, its value is zero. However, while gaining that knowledge I have also learned a lot of tools and methodologies that could be used elsewhere. Here is a short list:
  • How to verify data/code prepared by others before using them in your own project.
  • How to write good reports
  • How to manage engineers
  • How to design complex software
  • Matlab/Simulink
  • C/C++
  • Java
  • C#
These skills and tools can easily be translated to other domains, which makes them very valuable. Concentrate on what skills you will acquire while pursuing your topics of interest and make sure that they have some use other than your own narrow field.

Thursday, October 05, 2017

Çocuklara yazılım eğitimi

2014 yılında Facebook WhatsApp'ı 19 milyar dolara satın aldığından beri memlekette yazılım geliştirmeye ilgi arttı. Bu aralar çocuğu olan kişiler benim yazılımcı olduğumu öğrendiklerinde "çocuğumu programlama konusunda nasıl eğitebiliriz" diye soruyorlar. Okullar programlarına kodlama dersleri koymanın telaşı içindeler.

Daha da büyüyecek bir alan olması nedeniyle yazılıma yatırım yapmak mantıklı. Çocuğun yaşına göre müfredatın, özellikle de ilk adımın nasıl olması gerektiği mühim. Önerim öncelikle velinin temel becerileri edinip çocuğuna yol göstermesi, yoksa çocuğun gelişimi tesadüflere kalıyor. Eğitim/ilham başkalarına devredilemez.

Lego Mindstorms güzel bir platform. Veli internetten Lego Mindstorms'un nasıl kullanılacağını öğrenmeli, sonra seti alıp ilk adımları kendi atmalı. Böylece çocuğuna yardımcı olabilir veya göndermeyi düşündüğü okulun/kursun kalitesini değerlendirebilir. Kendi öğrenmeden Mindstorms'u alırsa çocuk altından kalkamayabilir ve birkaç gün içerisinde sıkılabilir. Birkaç ay yardımcı olduktan sonra ilgi uyanmışsa çocuk giderek daha bağımsız devam edebilir.

Geçen yıl 12 yaşında bir tanıdığa Lego Mindstorms ile yazılım/robot eğitimi verdim. Eğitimde yazılımın yanısıra hataların öğrenme fırsatı olduğunu, bir işi başkalarına anlatabilmenin önemini ve yönetimlerini vurguladım. Birlikte çektiğimiz video bu linkte.

Güncelleme 27 Kasım 2017: Neredeyse tüm çocukların oynadığı Minecraft üzerinde Scratch ile program yazarak kodlamaya giriş:

Monday, September 18, 2017

Using images as resources in C# WPF

To use images as embedded resouces in C# WPF (Visual Studio 2017):
  1. Go to Project properties / Resources tab.
  2. At the tab top menu, click the down arrow to the right of Add Resource and click Add Existing File.
  3. Select image files on disk. The files will be displayed on resouces tab. You will also notice that a Resources folder has been created in solution explorer.
  4. In solution explorer, select the image files under Resources and right click, select Properties
  5. Change Build Action to Resource
  6. In your code: MyImage.Source = new BitmapImage(new Uri("pack://application:,,,/Resources/" + imgFileName)
  7. Clean Solution, Build Solution
  8. Now your code will run correctly. The image files will be embedded into the exe file, you won't need to distribute the images as separate files.

Friday, September 15, 2017

Software in the loop simulation

When developing software that will interact with electronic hardware, it is important to have abstractions for hardware, i.e. software modules that mimic hardware behavior. Electronic components can have many problems that are not related to logic, e.g. loose components, need to reset from time to time, delays and data loss. With software abstraction, you can test 90% of logic without dealing with hardware complexities. Even better, you can easily automate your tests.


Once you establish that you are 90% correct, you can start testing with hardware to tease out the remaining 10% of bugs (real time issues etc.). The cost of software abstraction is that your design gets a little more complex and you need to write a couple of additional classes and interfaces to simulate the hardware logic. But it is well worth the effort because it decouples you from the hardware, saving you lots of work and frustration.

As an example, I am currently developing a desktop software that will be used to configure an electronic card through the serial port. I wrote a CardSimulator class and implemented the response of the electronic card to inputs from desktop in that class. On startup, my software can be configured to run in simulated or real mode, real mode meaning working with the real card. I have unit tests providing me with confidence that the desktop side of the equation is correct and when there is a problem, we only need to focus on the electronic card side.

Sunday, July 16, 2017

CI for C# projects: AppVeyor

I tried Travis CI for continuous integration of a C# WPF project and it failed because Travis CI uses Mono and that is not sufficent for WPF apps. So I tried AppVeyor and was successful. My minimalist appveyor.yml file is as follows:
image: Visual Studio 2017
configuration: Debug
before_build:
  - nuget restore

Friday, July 07, 2017

Difference between novice and experienced engineers

It has been said that good/experienced software engineers can be several orders of magnitude (i.e. 10x-100x) more productive than novice engineers. I think the main productivity improvement is the fact that experienced engineers are much better at detecting/avoiding bullshit:
It’s far more dangerous to assume people know what they’re talking about, than it is to assume they don’t and let them prove you wrong.
When I was a fresh engineer, I naturally did the work given to me by other people, without questioning if it was worth doing. As I got more experienced, I realized that most of the time people don't have thought through the job they are assigning to others. This results in getting stuck in dead ends, doing things that have no importance which result in lost time and motivation.

Nowadays, I am in my 20th year as an engineer. I pride myself in my analysis skills, ability to see signal through the noise and avoiding waste of resources. A lot of times I finish a job by demonstrating that the job in question is worthless/wrong. If you compare my lines of code per unit time, a novice might produce more. But that usually means producing garbage at a faster pace, creating more problems than solutions. That is why experienced people are worth their weight in gold.

Saturday, July 01, 2017

Using Travis CI for Java Projects

When using Travis CI for automated testing / continuous integration of Java projects created with the NetBeans IDE you have to do the following:
  • On NetBeans, Project/Properties/Sources make Source/Binary Format equal to JDK7. If you use JDK8, Travis CI ant will fail.
  • You must have at least one JUnit test.
  • You have to copy Junit and Hamcrest jars to project directory and add them under Libraries in NetBeans. If you leave them under Test Libraries, Travis CI won't be able to find them.
  • Commit changes to GitHub.
  • Add .travis.yml file to your github project root with the line language: java
  • Go to Travis CI and sign in with your GitHub account. 
  • Click on your Travis CI profile picture / Accounts. Your open source GitHub projects will be listed.
  • Enable your project for CI by clicking the button to its left.
  • Click on the project to go to build page which kicks off build.
  • If you have done everything correctly, you should see a success message at the end of the build page.
  • To add the build banner on top of build page, click on the banner, select Markdown and copy the link. 
  • On your GitHub page, edit README.md, paste the link and commit. 
You can see the build results for my snake project here.

Tuesday, June 20, 2017

JIRA for project finish time estimation

Software projects are notorious for their schedule overruns. One of the reasons is that project finish dates are usually based on wishful thinking, wich does not even make those dates nano-percent dates.

Herewith I propose a data driven approach using JIRA: The most important thing is to have enough issues in place to make statistical analysis meaningful, let's say at least 30 issues. After that, we need to make sure that we close issues faster than we create them. Only after we satisfy these two conditions can we talk about reasonable finish time estimations.

Below is my proposed project status page that updates whenever issues change, i.e. you do not need to ask people about when the project will finish or what the completion percentage is, it is all done instantly and automatically. The manager will always have up to date estimations. Only after your team has sufficient practice with this system should you worry about velocity/burndown etc. charts.


Calculation of parameters:

Enough issues = Total nb of issues > 30 ? Green : Red

Issues decreasing = (nb of closed issues in last 3 months) > (nb of created issues in last 3 months) ? Green : Red

Avg. nb of issues closed per month = (nb of issues closed) / (nb of months passed from project start)

Months required to finish project = (nb of open issues)/(avg. nb of issues closed per month)

Estimated finish date = Todays date + Months required to finish project

Estimated completion = (nb of closed issues) / (total nb of issues) *100

Wednesday, April 05, 2017

Modular design

The most important software design principle is separating the system into independent modules. By modules, I mean separately compilable units that can be "easily" unit tested. Benefits of modular software design:
  • It will be harder to introduce unintentional side effects like
    • Adding dependencies to code not related to the module.
    • Delete code that is not related to the module.
  • Modules can be designed separately which decreases mental load.
  • Each module will have its own unit tests, when unit tests are run, not the whole project but only that module will be built.
  • Each module will have its own string resource file instead of a giant resource file for the whole project.
  • Bad design (usually due to inexperience, time pressure or plain laziness) will be limited to the module in question and won’t be a project wide cancer.
  • There will be less merge problems.
  • Portions that have higher security requirements can be dealt with locally in modules instead of affecting the whole code.
  • Being able to test modules independently helps in identifying bugs like memory leaks.
  • It will be easier to define and assign tasks to developers. As long as the inter-module interfaces are clearly defined, integration will be easy.
  • It will be easier to estimate time and resource requirements.
  • If will be easier to reuse the module in other projects.
  • A module might be used standalone to do some tasks that does not require the whole system to be online.
  • If MVC was used, desktop GUI can be quickly converted to web front end and logic can be used as back-end and the whole application can be converted to a web application.

Friday, March 31, 2017

Android client connecting to java server example

As a case study on how to use a VPS, I bought a VPS from Amazon LightSail, wrote a server in Java and a client in Android:

VPS:
  1. I selected the preconfigured Ubuntu option. It has no GUI, you have to do everything on the command line.
  2. I created a static IP. 
  3. In order to access it freely (ping etc.), go to networking tab, under firewall add “All TCP+UDP”
  4. On Ubuntu, I installed OpenJDK8:
    1. sudo apt-get update
    2. sudo apt-get install openjdk-8*
  5. I installed Putty to use as an SSH terminal
  6. I installed Filezilla for file transfers.
Server software:
  1. Wrote server in Java on my laptop and transferred the code to VPS via FileZilla.
  2. I used gson for object-JSON conversions. The latest gson jar file can be downloaded from here.
  3. I wrote a java build script in Ubuntu using nano.
    1. Note that if you write the script in Windows and transfer it to Ubuntu, it might not work because line endings in Windows and Linux are different.
    2. To make the file executable: chmod +x filename
  4. I built and run the server. I prepared a shell script and ran it like so: ./script.sh
  5. To make sure that server keeps running when I disconnect putty, I used nohup and disown
  6. Update 15.10.2017: To keep nodejs server running do the following: 
    1. nodejs index.js> stdout.txt 2> stderr.txt &
    2. Do not close putty by clicking the top right "x", type exit.
  7. To kill a process
    1. Note PID by running ps -ef (or better, pgrep <process name>)
    2. Kill process with sudo kill PID

Android client software:
  1. Wrote client in Android Studio.
  2. I used gson for object-JSON conversions.
  3. I had to use AsyncTask for connection, otherwise I get NetworkOnMainThreadException
The source code for java server and android client is on GitHub.

TODO:
  • Server: Handle multiple simultaneous connection requests

Saturday, March 18, 2017

Dealing with legacy Matlab code

A friend of mine has a project that works in Matlab Simulink 2007. He recently opened that simulink model in Matlab 2010 and could run it from Simulink without problems. But when he tried to transfer that model to xpc target for real time integration, the simulation hung and became unresponsive.

He and his team tried to find the problem by disabling Simulink blocks one at a time and in the end found the block that was causing problems. When they compared the 2007 version with the 2010 version, they could not find any differences which means that there is probably a backward compatibility issue that affects real time deployment. They had to manually recreate the whole model (more than hundred blocks) manually in Matlab 2010. Doing manual editing in code always carries the risk of breaking something. You might not know it unless you do tests with high coverage.

My first advice was to create a unit test framework so that you are sure that the new version yields the same outputs for the same inputs. A good strategy is to write scripts that vary inputs using unform distribution, run simulation on xpc target for every input, record outputs (or timeout when simulation hangs) and then compare outputs with previous results and create a report that highlights differences.

My second advice was to automate the update-to-newer-Matlab process by writing a program that automatically creates a new Simulink mdl file from the old one, beause it is easy to parse the mdl file which has an xml-look. Steps:
  1. Takes as input the depth of parsing. 1 = Copy the whole model. 2=Go one step deep and create separate mdl files for blocks in that depth 3=Go 2 steps deep etc.
  2. Parses the old Simulink mdl file and extracts the blocks
  3. Creates an mdl file in the new version format.
  4. Inserts the blocks into new mdl file(s)
  5. Runs unit tests on xpc target to see if the two versions have the same outputs (i.e. they are within 1e-15) neighborhood.
With this program, my friend can first check if for input depth =1, the model passes tests. If not, he can increase the depth until he can isolate the problem. Increasing depth on error could be automated too.

Saturday, March 11, 2017

Intelligence tests

I don't have much faith in intelligence tests. My definition of intelligence is successful adaption to new circumstances. Adaption and persistence determine your material success. How can you hope to test adaption, especially the social component in an hour long test?

Recently a friend of mine told me that his child's test score indicated his child being super intelligent but my friend didn't value the result much, although it was better than having an imbecile score. Our conclusion is that such tests mainly measure how much care the child was given by the parents. If parents have done a good job of parenting, I suspect any kid who does not have a mental condition will score highly in those tests.

Unrealistic proposal requests

Besides software development, my responsibilities also include assessing and answering simulation related proposal requests from other firms. Since simulation is usually poorly understood, the requester puts in sentences like "the simulation shall be realistic" without knowing how vague and overly ambitious that is.

Without going into philosphical details about what is "realistic", I can say that it involves a lot of real world testing and modeling. The budget that is available is usually less than 1% needed to fulfill realism. In that case we have two choices:
  1. Prepare a proposal that takes every possible risk into account and demand a billion dollars (same as saying NO).
  2. Think about what can be done with the given budget, write down your assumptions and limitations and present that as a proposal. This is the way to go if you want to avoid suffering.
The major cost factor in a simulation project is integration and testing. Your first question to any proposal request should be "how will you know that our simulation works as you expect". The answer will determine not only how much testing is needed but also how detailed your models should be. It is like unit tests driving design. If you don't get a satisfactory answer, write down in your proposal how and where you will test it.

Sunday, March 05, 2017

Software manager reading list

A few days ago, a manager friend of mine asked me for advice about what to do in a project that is near crisis, i.e. it has missed a deadline and looks like will be over budget.

Crises are common in software projects because their complexity usually is beyond our time and resource prediction capabilities. I offered my friend the following books and recommended reading them first before demanding that his team starts working overtime:
These books will ground him in reality and make him realize what kind of a mess he is in. I also told him that he should only tell his superiors the reality if he is certain that they are interested in the truth. A lot of times people want to hear only good news and are content with ignorance until it hits.

If you sense that your superiors want to live in fantasy land, never tell them the bad news because you will be labeled as a pessimist. Even after your predictions prove to be true in the end, you being labeled as a nuisance won't be washed away.

These books will help you understand what is going to happen and why. You might not be able to change the outside world, but you will have less stress inside, not feeling any guilt, knowing that you did the best you could.

Monday, February 27, 2017

Probability: Difference between "exactly..." and "at least..."

In probability, I have to pay close attention to the problem statement because I easily make errors. As an example, let's say I toss a fair coin n times. What is the probability of
  1. Exacly 1 head
  2. At least 1 head
The figure below has n as the x axis and the two different probabilites as y axis. As you can see, the "exactly 1 head" probability decreases with n while the "at least 1 head" probability increases with n.

Matlab script for generating the plot:
n = 1:16;
pExactly1 = n./2.^n;
plot(n, pExactly1); grid on
hold on
pAtLeast1 = 1-(1-0.5).^n;
plot(n, pAtLeast1)
legend('exactly 1 heads', 'at least 1 heads')
xlabel('n'); ylabel('p')
Note that you can also get probability of at least 1 heads with this sum: pExactly1Head + pExactly2Heads + ... + pExactlyNHeads, where
pExactlyKHeads = nchoosek(n,k) * ph^k * (1-ph)^(n-k)
Even simpler: 1 - pExactly0Heads = 1 - nchoosek(n,0) * ph^0 * (1-ph)^(n-0).

For more information, visit Khan Academy.

Car IoT

As aircraft have become software with wings, so are cars on their way to become software on wheels. Self driving and IoT have accelerated this trend. Yesterday, when I was talking with my friends we envisioned a design that could improve driving comfort: Cars could collect vibration data measured by their sensors and transmit it together with location information to servers. This data would indicate road quality. The server could process and broadcast data to all cars. Other cars could adjust suspension settings in advance that would maximize comfort on that road.

With this technique, only the first few cars that collected data would suffer and the rest would benefit. Data would be current at all times. I think Google Maps uses a similar approach to guess traffic congestion by collecting data from smartphones.

To collect data we need:
  • Vibration sensors on car.
  • Access to internet and access to central data server
To adapt to road conditions we need:
  • Adaptable suspension on car (i.e. actuators and sensors)
  • Access to internet and data server by car.
If we could protect privacy, we could automatically tap into the intelligence of the swarm.

Note to self: Do a simpler version of this concept as a hobby project.

How to evaluate personnel

It would be easy to evaluate performance if everybody (including the evaluator) was of high quality and there was no Dunning–Kruger effect. Your only criteria would be years of experience. Unfortunately, real life is more complex than that and there are large differences between employees.

The next best thing might be to have three bins: low, average, high. The low bin might be for fresh graduates who just started to work. The high bin consists of people who are highly productive and experienced.

You divide your staff according to their general level into these bins. Each bin has a lower and upper salary bound. Then you rank the people in each bin according to their productivity. I know that realistically measuring productivity is a tough problem but there is no escaping from it. For example, you have to decide if the person with 3 months of overtime had really productive overtimes or was just producing 8 hours of work in 12 hours. In each bin, at the top of the pyramid should be people with high productive overtime and at the bottom should be persons with no or low productive overtime. That way, you assure that everybody gets their fair share and productive individuals get their extra.

Tuesday, February 21, 2017

Android development tips

When developing Android apps with Android Studio, I get strange errors from time to time, whose solutions were not obvious to me:

Problem: When you create a Google Maps demo and run it using an emulator of API level 24 (e.g. Nexus 5 API 24), the app opens on the emulator with the message "...won’t run unless you update Google Play services".

Solution:  In \app\build.gradle file, com.google.android.gms:play-services version might be 10.2.0. Change it to 9.8.0 and your app will run fine.

Problem: In Android Studio, when I try to generate signed APK, I get "android execution failed for task validatesigningrelease" error.

Solution: Make sure you specify full path of key file.

Problem: When I try to generate signed APK, I get “Execution failed… method ID not in [0, 0xffff]: 65536” error.

Solution: If you are using Google Maps, limit the number of libraries you use by replacing in your gradle file the line compile ‘com.google.android.gms:play-services:9.8.0’ with
compile 'com.google.android.gms:play-services-maps:9.8.0' and
compile 'com.google.android.gms:play-services-location:9.8.0'

Sunday, February 19, 2017

Better WhatsApp Group Chat

Group chat is widely used in WhatsApp. As the size of the group gets bigger, communication is dominated by few people who usually generate a lot of noise. I had to leave groups because I was fed up with the endless nonesense of these few people. Arguing with them makes matters worse. Almost everyone I know has similar experiences.

I can silence a whole group in WhatsApp so that I do not get notifications. What I need is the ability to mark people in a group as important and unimportant. Messages from people that I mark important should be easy to spot when I scroll through the messages. They could have yellow background and bold font. Messages from people I mark as unimportant should be shown collapsed and become only visible in full when I click on the message. That would allow me to quickly see important messages.


Friday, January 20, 2017

My first Android app: Rescue Map

A friend of mine is a member of the search and rescue organization AKUT. He asked if there is an app that could draw the estimated victim region on a map according to victim category.

The category specifies at what speed the victim is expected to be moving. The app has to have offline map capability so that it can be used when there is no reliable internet connection.

I did a quick search on Google Play and could not find anything that did the job. So, I decided to write it myself as a first project in Android. The source code is on GitHub. I don't plan on publishing it to Google Play until my friend says that the app is really useful and others could benefit from it.

Racism test

Compared to older times, open racism is in decline. It takes on more light forms like "I love my culture/people". There is nothing wrong in loving certain things or having preferences. The critical point is how you view others: Do you just want to be left alone with your pure clan or do you think you also have a "natural" right to decide on others destinies because you are part of the master race/gender?

Likeable vs logical

In both professional and social interactions, being likeable is much more important than being right. I am amazed how much this fact is ignored by otherwise intelligent people. If people don't like you enough, they won't even listen to you which means that you are wasting your time constructing arguments. People will be meaner to you. Your name won't come up when there are opportunities or positions to fill. In short, your quality of life will suffer.

How can you be more likable (in order of difficulty):
  1. Smile and greet a lot.
  2. Don't talk if you have nothing nice or productive to say.
  3. Avoid ad hominem attacks and generalizations. Example: "You always do this because your are an emotionless prick!"
  4. Find mutually beneficial solutions to problems, even if it means to settle on a less-than-optimal solution.
  5. If you have to argue, keep calm and argue with unambiguous data. This implies that you have to collect data before the argument.
  6. Be sympathetic to others. People rarely mean exactly what they say. There are different dynamics at work that makes them behave as they do. If you take their words at face value, you might think that they are dumb or crazy. They are not. Their feelings might be hurt by something else, they might be tired etc. 
People have flaws, don't treat them like computers. The next time you are about to label others stupid because they are not convinced by your perfect arguments, ponder if they like you or not.

Book: Dark Sun: The Making Of The Hydrogen Bomb

Dark Sun: The Making Of The Hydrogen Bomb, Rhodes, Richard, documents the history and politics of hydrogen bomb development both in the USA and the Soviet Union. For the story of atomic bomb development see my previous post.

Book: Command and Control

Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion of Safety Schlosser, Eric, details how fragile nuclear weapons safety is.

Book: The Dictator's Handbook

The Dictator's Handbook: Why Bad Behavior is Almost Always Good Politics, de Mesquita, Bruce Bueno, Alastair Smith, explains why people in power might do seemingly irrational things and why loyalty is much more important than competence.

Similar previous blogpost: link 1, link 2

Related video 1:


Related video 2:


Related video 3:

Book: How to Fail at Almost Everything and Still Win Big

How to Fail at Almost Everything and Still Win Big by Scott Adams, is about trying different things, failing often, learning as much as possible from each failure and be ready when luck finds you. Highly recommended...