Select Page
Fig. Life Lessons from Gradient Descent

It’s been a long time since I was introduced to the Gradient Descent algorithm. Basically, gradient descent is an algorithm in optimization theory which helps us find out the most optimal solution to a function. It is based on a simple idea, rather than finding out the best solution to a problem immediately, move in the direction that seems to take us to the optimal solution. I have made a small list of philosophical thoughts that compare Gradient Descent to real life struggles.

1. Drastic Changes NEVER WORK

Do you remember the time when you told yourself, “I am going to work for the next 10 hours continuously and get this thing done!” and then it didn’t happen? That time when you went to bed at 2:00 in the night and thought of waking up at 6:00 in the morning to complete the work, only to wake up much later?

Gradient Descent teaches us to make changes proportionally(negative of gradient or derivative) rather than drastically so that we reach the optimal solution one day instead of overshooting and making our condition even worse. I know tackling a problem head-on is tempting and it gives us an adrenaline rush when we think about it but, that does not give you long-term results. Everyone wants to make a definitive to-do list and strike off items one by one, but habitual and long-term changes are not made like that. Small steps in the right direction are the only way to succeed in the long run.

2. The journey is IMPORTANT not the state

In life, we see many people achieving the same things and we judge them all using the same criteria. Most of the people will judge a billionaire by looking at his wealth, his house, his car and so on, but they will never see the journey has been through to reach this successful state. Some people are fortunate to be born with some facility and some are not. If both have reached the same state, that does not mean both took the same path and experienced the same journey. For example, one of my friends was born into a rich family with a million dollar family business and another friend was born in a middle-class family where only his dad earned money through driving a cab. If both of them are now running a company with same net-profit and turnover, then it would be foolish of us to think that they both have gone through the same struggles.

3. There can be better states, just keep WORKING

Fig. Different States in Life/GD Algorithm

As shown in the image above, there are two states a person can achieve. In our case, assume lower the better. So State 2 is better than State 1. When a lazy person is on State 1 he/she might think that he has already reached the lowest point and won’t make efforts to improve. But an active and ambitious person always tries different paths and keeps improving no matter what; even if he has to take a path is full of hardship. But soon the ambitious person finds another slope that goes further downhill that his current state (i.e. State 1). This is when he reaches State 2. Success is subject and can be deceiving, only the one who keeps trying different paths (stochasticity) gets to the most optimal state.

These are the three philosophical thoughts I relate to when I compare Gradient Descent to Life Struggles. If you have any other suggestion or example, comment below.