The OD Cost matrix does NOT give direct/aerial distances. It computes the network shortest paths. But, for performance, it does not return back the route geometries of the path taken and simply draws a straight line to indicate the pair for which the network cost is computed. It is designed to return a matrix of distances. If you need to get the route geometries as well then use the Closest Facility solver. It does what OD does but also returns the route shapes. Make sure to set the number of facilities to find to the number you have (destinations) as the default is to find 1.
Jay Sandhu