Research on road monitoring system has been actively conducted by using both machine learning and deep learning technique. One of our nearest goal in the framework of road condition monitoring system is to segment all road related object and provide a technical report regarding road condition. Our final objective is to develop a community participant-based system for road condition monitoring. As one of our task, in this research, we start with the segmentation of road and pothole. To conduct this task we proposed a semantic segmentation method for road and pothole image segmentation by using one of the famous deep learning technique U-Net. Various condition of road images were used for training and validating the model. The experiment result showed that U-Net model can achieve 97 % of accuracy and 0.86 of mean Intersection Over Union (mIOU).