Electroencephalography (EEG) signals have been proven to be effective in evaluating human's cognitive state under specific tasks. Conventional classification models utilized for EEG classification heavily rely on signal pre-processing and hand-designed features. In this paper, we propose an end-to-end deep neural network which is capable of classifying multiple types of cognitive workload of a vehicle driver and the context of driving using only raw EEG signals as its input without any pre-processing nor the need for conventional hand-designed features. Data used in this study are collected throughout multiple driving sessions conducted on a high-fidelity driving simulator. Experimental results conducted on 4 channels of raw EEG data show that the proposed model is capable of accurately detecting the cognitive workload of a driver and the context of driving.