The Sin of Generalization

Our esteemed, long-gone ancestors granted us an advantageous capability—acute visual pattern recognition. It was critical for survival to distinguish tree branches from pythons and good fruit from foul ones. Now, we train neural networks to do the same: recognize patterns, generalize, and produce new variations. The big difference here is that we teach models where to stop.

This post came out of my reply to one of those silly social network quiz posts.

You would not believe how complex some previous solutions were. The reasons behind the answer I gave are:

• The cans in the picture are empty (at least for me).
• The smaller one has a height close to the diameter (square shape in dissection.)

The answer itself was not entirely correct, so a lot of folks pointed out that:

• That you waste two liters of water when you tilt the can
• This method does not work for all shapes
• That angle is not a 45º
• You can't find an angle without extra tools
• That you should use something to mark half of the can and poor water into it

Does this sound familiar to you? Of course, it does because it's precisely how software engineers design program components, and the process never ends until their LBBs (Large Biological Brains) reach the stop word. It could be different for different people, but my magic words are:

I am proud to be friends with a few excellent developers and architects familiar with the sin of generalization. You can't avoid this, especially if you have to operate on limited information. You could create universal frameworks, implement hundreds of methods, or make your code run faster by 0.5 seconds. At some point, you should suppress your generalization cycle and ask yourself, "Is it good enough for this case?" The hard part is to give yourself the most sincere answer, and if the answer is "yes," break the look and hold improvements.

I'm not sure if the Agile Manifesto resonates with me or if it is the reason why Agile has taken over most modern development projects. But this is the only way you could create a complex project. The space race between SpaceX and NASA is the perfect example of that.

I'm about to stop my word generator and finish this piece with a few takeaways:

• Look for the most effective solution that fits initial conditions and assumptions.
• Make sure that your solution "for this case" is not final and could be adjusted in the future. But don't go far from the most efficient one if you have to.
• When more data is available or conditions change, rethink the solution and approach. In most cases, you would never do that.