If you just have a single problem to solve then fine go ahead and use a neural network. But if you want to do science and understand how to choose architectures or how to go to a new problem you have to understand what different architectures can and cannot do.