if you have two groups of points and you want only the shortest one back then the only way to do this is to solve for all with the OD Cost Matrix, pick the shortest pair and put that into a Route solver and solve it to get the route geometry or directions. You can also use the Closest Facility to solve all and then pick the shortest and get directions. But that may take much longer depending on how many points are in each group.
That is, there is no other way to know, without computing them all, which pair will give you the shortest path between two sets of points.
Jay Sandhu