Why is there so much confusion over ideas like fate and destiny?

In a Nutshell: The confusion that ensued for centuries because of al-Qadhaa wal Qadar is due to the adoption of logical principles laid down by Greek philosophers. The correct Islamic understanding of the subject can only be reached by removing the logical assumption attached to it. The subject of al-Qadhaa wal Qadar, otherwise known as Fate and Destiny, has been a topic of discussion for centuries. As the question implies, it has truly led to confusion among scholars. The three major schools of thought that discussed this subject were the Mu’tazilah, Jabriyyah, and the Ahlus Sunnah. The discussion kickstarted when Muslims encountered Greek Philosophy and debated them, topics like al-Qadhaa wal Qadar came about. The question that underpinned the discussion was laid. It went thus: does man control his actions, or is he compelled to carry them out? This then led to the assumptive principle for reaching a conclusion, which was: if man created his actions, then he controls them; and if he did not create them, he does not create them. The two Greek schools of thought reached their opinions banking on this logical principle. The Epicureans concluded that man controls his actions and, therefore, he created them. The Stoics concluded that man does not control his actions and thus, he did not create them. When Muslims came across this challenge, they approached it in good faith by abiding to the aforementioned logical principle, which says if man did not create his actions, he does not control them; and if he controls them, then it means he created them. The Mu’tazilah The Mu’tazilah were the first to address the matter. Their general opinion was that man controls his actions and, therefore, he created them. Their logic was that if man did not control his actions, it would be unfair to judge him on the Last Day. And it is not acceptable to describe Allah (swt) with injustice. They quoted verses like And nor is your Lord ever unjust (in the least) to (His) servants.[Fussilat: 46] And, It is not Allah who wrongs them, but they wrong their own souls. [At-Tawbah: 70] The Jabriyyah The Jabriyyah were the second from the Muslims to address this subject and in opposition to Mu’tazilah they concluded that man does not control his actions, he floats like a feather to where the air takes it. This is because man did not create his actions because Allah (swt) said But Allah has created you and your handiwork! [As-Saffat: 96] Hence, there is no way man would control his actions, since he did not create it. Ahlus Sunnah As for the Ahlus Sunnah, they came with an opinion saying “It came out from between the refuse and the blood, pure milk palatable to the drinkers”. Their opinion, in summary was that man has Kasb Ikhtiari (Free-will). By this they mean man did not create his action, however, he initiates the creation of the action when he intends to carry it out. This is to say that Allah (swt) creates the action when man intends to carry it out. Therefore, it was man who initiated the intention; however, he did not create it. Without dabbling into technicalities, the opinion of the Ahlus Sunnah actually does not differ with the opinion of al-Jabriyya. This is because the expression of Kasb Ikhtiari does not mean man is not compelled to carry out his actions. As a matter of fact, it means he is compelled to carry out his action. They imply that by using different expressions without realizing. The Source of Confusion It is clear that all of these opinions were reached by applying Greek logic and that these schools of thought tried ‘Islamizing’ the opinions that were already reached by the Epicureans and Stoics by associating them with Aayats. In essence, they were source-mining. From this we understand that the reason confusion ensued in the topic of al-Qadaa wal Qadar is from the approach itself. The Muslims should not have adopted the principles laid by the Greeks, because this would, as it did, not allow them to apply Islamic principles. Their intention was pure but their application was not. Although, this question asks about the reason there was confusion, I will briefly explain the subject in a correct manner. The Correct Islamic Understanding of al-Qadhaa wal Qadar The correct understanding of al-Qadhaa wal Qadar can only be reached when we understand the correct basis of the discussion and, therefore, the Greek logical principle must be put aside. In summary: The topic has to do with man’s actions. And these actions fall in two spheres: one sphere controls and dominates man while the other he dominates and controls. The sphere that dominates man is in two categories: the law of universe and the outcome of an action. The Law of Universe is like ‘man cannot fly’ and ‘man cannot live under the sea’ in his natural form. This is because Allah (swt) created man in a way that he cannot do such things. Man cannot phase through a solid wall because by natural law his body molecules can’t reshape to a form that allows him to do that. This is what it means by ‘the sphere that dominates man’ in relation to the Law of Universe. It does not mean it makes him do things on his own, rather it means they restrict him and when he tries to break them a consequence befalls him. For instance, if man tries to jump off a tall building to fly, he falls down hard and gets hurt or even die. The Law of Universe forces man to consider what he can and cannot do. Knowing he cannot live under sea, it is not an option to live in the sea except if it is a submarine, which does not allow water to enter. In any event, he cannot live where he breathes through water. As for the outcome of man’s actions, it is like shooting a gun and hitting someone without knowing the person was there. Or throwing a stone that hits someone or something without knowing it is or he is there. Or being hit by something without knowing it is coming. All in all, it was initiated by man – either he initiates it and it befalls someone else or someone else initiates it and it befalls him; depends on which end you are at. The Law of Universe and the outcome of man’s actions is what is called Al-Qadhaa. As for al-Qadar, it is the attribute found in objects and instincts. These attributes do not influence or force man to carry out an action. Man chooses to use them the way he wishes. Hence, fire does not force one to burn down houses. Man chooses to cook or burn down places with it. The knife does not force one to cut a chicken. Nor does water forces one to wash. However, we choose to use them based on the attributes Allah (swt) placed in them. Those attributes are referred to as al-Qadar. It should be noted that the Al-Qadar discussed here is not the lonely al-Qadar, which means something else. This is a description of al-Qadar in relation to the topic of al-Qadhaa wal Qadar. Conclusion The confusion that ensued for centuries because of al-Qadhaa wal Qadar is due to the adoption of logical principles laid down by Greek philosophers. The correct Islamic understanding of the subject can only be reached by removing the logical assumption attached to it. Reference(s) Taqiuddin an-Nabhani, Shakhsiyyah Islamiyyah (Islamic Personality) Vol 1 Taqiuddin an-Nabhani, The System of Islam