i finally find the solution
i write the code to help anyone want do this conversion
this is the code (using c#) to convert layer to KML:
//convert layer to kml
ESRI.ArcGIS.Geoprocessor.Geoprocessor GP = new ESRI.ArcGIS.Geoprocessor.Geoprocessor();
GP.AddToolbox(@"C:\Program Files\ArcGIS\ArcToolbox\Toolboxes\3D Analyst Tools.tbx");
GP.SetEnvironmentValue("workspace", @"C:\Maps\Maps");
GP.OverwriteOutput = true;
IVariantArray parameters = new VarArrayClass();
// assign parameters
parameters.Add(mylayer);
parameters.Add(@"C:\Maps\Maps\city.kmz");
parameters.Add(1);
GP.Execute("LayerToKML_3D", parameters, null);