For the same reason the world still uses the QWERTY layout, although more efficient ones have long been created. The masses are always inert, changes are created by small groups of initiators, followed by the masses (or not).
Strictly speaking, Esperanto has taken root very well. Esperantists live all over the world and actively use Esperanto for everyday communication, travel, business, romantic relationships, etc. Yes, he did not become dominant, like English, but that could not happen. Hitler defeated the Esperanto community in Europe, Stalin in the USSR, and after World War II the rise of the United States began, and with it the victorious march of the English language, which was so marginal. Esperanto was of no interest to anyone politically, and without it it would be naive to expect support from politicians.
Esperanto did not take root because there is no economic, political or military power behind it. Dot. Everything else is from the evil one. And the language itself is great. According to the famous polyglot Benny Lewis, even if not a single person spoke Esperanto, this language would still be worth learning. Firstly, no ethnic language brings brains in such order as Esperanto (well, perhaps, Quechua and Aymara). Ethnic languages teach first of all to cram, and Esperanto teaches to think first. And secondly, Esperanto is an excellent propaedeutic. Numerous experiments in different countries have proved that knowledge of this language greatly facilitates the study of foreign languages. For example, as a result of teaching Esperanto in Hungarian schools, it was found that for Hungarians, knowledge of Esperanto facilitates learning French by 50%, English by 40%, German by 30%, and Russian by 25%.
Why do you need it if you have English? By the time it appeared, English was already widespread, French for example, with which it could hardly compete. The entire civilized world has long been speaking English, but there will be no sense in uncivilized Esperanto. After all, knowing Esperanto, even in the most optimistic scenario, cannot replace knowledge of English. Even if the whole world decides to switch to Esperanto in international communications, English and American scientists and businessmen among themselves will still continue to communicate in their native language. And anyone who wants to study at Harvard and Oxford, trade stocks in London and New York, or even just read scientific articles in magazines like the American Journal of Something, without fluency in English will still be nowhere (just like now nowhere without knowledge of French in the former colonies, where ties with France are strong, and knowledge of Spanish in Latin (South) America).
And it is not so simple. The grammar is regular and simple, everyone learns very quickly, but even with a powerful word-formation system, almost all vocabulary will have to be crammed as hard as in other languages. If you speak a couple of European languages, then you can learn Esperanto without any problems, but the question is - do you need it in this case?