Why does von Neumann obstruct deep learning?

The operation of the brain has inspired von Neumann’s famous paradigm. How did the data- and event-controlled analog brain inspire the model of the instruction- and clock-controlled digital computer? One of his great ideas was to store the computer’s knowledge in the computer’s memory. Why loading a ready-made software program into its memory is the only way to teach the computer’s hardware to do something? Given that the instruction execution of the processor is so fast, why is the software-implemented training by machine learning so painfully slow and power-wasting? Is using another computing model, component, or algorithm with faster operation risk-free?