In this work, we introduce a new calibration method for a camera system comprising five Azure Kinect. The calibration method uses a ChArUco coded cube installed in the middle of the system. A new 3D optimization cost is proposed to overcome the IR camera noise and to enhance global 3D consistency of the captured model. The cost includes the repro-jection error and the point to plane distance. As a refinement stage, along with point to plane distance, a patch to plane distance is added in the cost to overcome the noise effect of the depth camera. The experimental results demonstrate that the proposed calibration method achieves a better reprojection error and more stable results in terms of standard deviation of the estimated pose compared to the state-of-the-art. In addition, the qualitative results show that the proposed method can produce a better registered point cloud compared to conventional calibration.