You are passing the incorrect type of data to the R script. It is intended for vector feature classes and not rasters. I believe that the ESRI script is specifically intended for point feature classes. If you want to stick to ArcMap you can try IsoCluster. However since the algorithm is based on means, if your data is non-normal you will not get a very good cluster solution and you may need to apply a data transformation to your rasters (i.e., natural log) before running IsoCluster.