If you go to the new BETA Vector Basemaps group, there is an option for some of them to open in desktop. When you select this, you get a .pkinfo file that downloads. When you click on the .pkinfo, it opens in ArcGIS for Desktop, but it does not do anything, and shows up with a red X in the table of contents.
Are the BETA vector basemaps not currently supported for desktop?
http://www.arcgis.com/home/group.html?id=30de8da907d240a0bccd5ad3ff25ef4a