If you just have a single problem to solve then fine go ahead and use a neural network. But if you want to do science and understand how to choose architectures or how to go to a new problem you have to understand what different architectures can and cannot do.
My parents were drawn to the idea that there was space and opportunity in Australia. For the meagre sum of £10 you could sail your entire family out to Australia so that's what my father chose to do.
