The depth maps captured by sensors always suffer from low resolution and random noise. Recently, by introducing the guidance from the color image, deep convolutional neural network (DCNN) shows significant improvements for depth map enhancement. Howe