The finite element analysis (FEA) of magnetic field generally requires a lot of calculation time. Especially, design optimization methods such as the level-set method with FEA result in large computational effort to find better solution. In this paper, we propose a novel method of precisely and quickly reproducing the conventional optimization steps by means of Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM). The developed method enables us to implement high speed search of solution, which means the possibility of effective optimization with various initial conditions for better solution. Finally, we evaluate calculation time and computational accuracy of the proposed method by using a magnetic circuit design model.