Breast cancer is the most common type of cancer among women, it has the highest death rate among all types of cancer. Pre-surgical (neoadjuvant) treatment can improve the patient’s prognosis, but it is not possible to predict whether the patient will respond to treatment. Previous studies were conducted to find characteristics capable of associating neoadjuvant treatment with patient response, some using conventional radiomics and others convolutional networks, mostly using private databases with few examples, those studies are non-reproducible. In this work, a deep learning model was proposed using images and clinical data from a public database (Duke Breast Cancer MRI), capable of extract characteristics of breast MRI images, and associate the attributes with prognosis. Using quantitative analysis of results generated from empirical experiments, it was concluded that the proposed model is capable of classifying patients that achieved complete response. The results showed an accuracy of 60% and AUC (area under the curve) of 0.63.