Over the last few years, there’s a lot of conversation about all the possibilities of machines doing tasks that typically require human intelligence, and even outperforming humans at those tasks. Such tasks include object recognition, speech translation in real time, disease and tumor diagnoses, and even music composition. Thanks to powerful algorithms, machines can now learn by experience, just like we humans do and interact with their environment in an autonomous manner. Those algorithms belong to a field in Computer Science called Machine Learning.
Deep Learning is subfield of Machine Learning where algorithms mimic the workings of the human brain in processing (large) data for use in decision making. These data can be text, voice, image, video, etc. Deep Learning consists of various (deep) layers of neurons (Figure 1), with each layer receiving inputs from previous layers, and passing outputs to further layers. An alternative way of thinking about it is to think of it as one massive Artificial Intelligence function which takes inputs and arrives at a final output (or decision).
Figure 1 Deep Neural Networks
Deep learning can help in both clustering and classification tasks. They help to group unlabeled data according to similarities among the example inputs, and they classify data when they have a labeled dataset to train on. An example of clustering includes Google News where articles are automatically grouped by topics, and an example of classification includes image classification where an input image is assigned one label from a fixed set of categories (e.g. recognizing whether the image is that of a cat or a dog). Now let’s focus on Binary classification problems where data samples belong to two known groups (or classes), resulting in a yes/no decision (e.g. classifying an email as spam or not spam).
In a binary classification problem, class imbalance occurs when one class, the minority group, contains significantly fewer samples than the other class, the majority group. Those in the Data Science field know that learning from these imbalanced data sets can be very difficult, especially when working with big data. A concrete example is the medical diagnosis task of detecting cancer, where the majority of the patients are healthy and detecting cancer is of greater interest. Class imbalance causes learners to over-classify the majority group and misclassify the minority group. For those interested in knowing exactly why this happens, I recommend reading this paper.
Despite intense works on imbalanced learning over the last two decades there are still many shortcomings in existing methods and problems yet to be properly addressed. Some approaches to address this problem include Data-level methods (e.g. two-phase learning and dynamic sampling) and Algorithm-level methods (e.g. Output thresholding, adjustment of reward function in deep reinforcement learning and very deep neural networks). Readers can refer to this survey for more details. Algorithm-level methods are better equipped for big data problems since no change to training data or pre-processing is required. Very deep Neural Networks, for instance, require increasing the neural network depth (e.g. number of hidden layers between input and output). Examples of very deep Neural Networks include the Inception Networks and Residual Networks, which deal with dozens of layers. As depicted in Figure 2, increasing the number of layers allows the network to better separate classes. Basically, more layers means, more non-linearities applied to data, leading to a better establishment of the decision boundary between classes.
Figure 2. Separation power of Deep Neural Networks
On another aspect, adding more layers to the neural network leads to explosion in network and model complexity, while an insufficient number of layers may require an exponential size of neurons in one single layer. This problem gets even worse in the case of severe class imbalance: Imagine one example in the minority class for hundreds, thousands, or millions of examples in the majority class!
Now that we acknowledge the complexity of designing a Deep Neural Network that correctly separates classes in the case of severe class imbalance, I want to go back to my previous post, where I argued that all humanity (big) data is recorded in a “clear book” and that all this data will be used by our Creator to make an ultimate decision that would determine our fate as human beings. In fact, this decision is binary: either a zero (Individual goes to Hell) or a one (Individual is saved from Hell). [Sorry to put it this way; I simply reason in a scientific way!].
As a data scientist, I couldn’t help but think of the complexity that some Deep Neural Network would have if it were to take this ultimate decision by itself, given our records as input! The data that would be fed into this Deep Neural Network (i.e. our deeds!) would naturally carry a certain (positive / negative) reward whose value is unknown to us. Indeed, even though the overall decision is binary, the rewarding strategy isn’t. In other words, it’s not a kind of: “Individual does badly, it gets -1. Individual does well, it gets +1”. Good deeds and evil ones aren’t attributed the same reward (and there lies the divine mercy!). In fact, the reward of good deeds can be doubled and evil deeds can be replaced with good one, as mentioned in verse 70 of Surah Al Furqan:
إِلَّا مَنْ تَابَ وَآمَنَ وَعَمِلَ عَمَلًا صَالِحًا فَأُولَٰئِكَ يُبَدِّلُ اللَّهُ سَيِّئَاتِهِمْ حَسَنَاتٍ ۗ وَكَانَ اللَّهُ غَفُورًا رَحِيمًا
Which can be translated into:
“Except for those who repent, believe and do righteous work. For them Allah will replace their evil deeds with good. And ever is Allah Forgiving and Merciful”
In a very extreme case (severe class imbalance!), one good deed inside millions of other evil deeds can change the overall decision, as stated in Surah An-Nisa, verse 48
إِنَّ اللَّهَ لَا يَغْفِرُ أَن يُشْرَكَ بِهِ وَيَغْفِرُ مَا دُونَ ذَٰلِكَ لِمَن يَشَاءُ ۚ وَمَن يُشْرِكْ بِاللَّهِ فَقَدِ افْتَرَىٰ إِثْمًا عَظِيمًا
Which can be translated into:
“Verily, Allah forgives not (the sin of) setting up partners (in worship) with Him, but He forgives whom He wills, sins other than that, and whoever sets up partners in worship with Allah, has indeed strayed far away.”
Now I want you to think of what kind of Deep Neural Network would withstand such a complexity and still accurately classify instances. Could it be some INFINITELY Deep Neural Network that surpasses all human and machine intelligence?
If you made it till the end of this post and carefully followed my reasoning, you might also think of how fascinating the neurons in our brain work and how amazing they can make connections and draw some “uncommon” analogies and conclusions. This is the power of thinking that no algorithm or machine would ever be able to have!
#Enlighten_your_heart_and_your_mind_will_be_enlightened
