AGS 10.1 Python SHP with non-ascii chars  append to File GDB or SDE  => error

Discussion created by radekp on Apr 22, 2013

I'm facing a serious problem doing simple task - have a arcgis model (10.1), which is just taking shapefile and appending it into file geodatabase feature class or into sde (10.0, oracle 11.2)

Shapefile contains some national chars, until now tried win1250 and utf-8 versions, both worked perfectly inside Arcgis for Dektop (10.1), ldid byte in dbf file is set correctly, cpg file is also OK, all characters (their hex representations) are OK (I have checked its codes with win 1250 and utf 8 chars tables, both versions are correct, so I'm pretty sure the problem is not shapefile itself.

Executing model from within Arc Catalog (10.1) is OK, result is perfect without any error in character representation.
Executing model by python (2.7) at desktop (10.1, win xp cz) is OK too, result is the same as if executed from Arc Catalog

But - executing model by python (2.7) at server (10.1 sp1, win2008 en), using same python script, same shp, same target SDE or File Geodatabase - results in strange behavior - some (not all!) non ascii characters are replaced by their near (non diacritical) ascii equivalents (for example "?" is stored as "u", "??" as "r" ... but - also non-ascii "š" or "?" are stored correctly)

It has nothing to do with fonts or displaying attribute table at all, data really is malformed ... but only if executing model by python at Arcgis for Server.

I have tried to change windows cmd shell encoding (chcp), I labored with various python default encodings - but it has zero effect at all. Only changing ldid byte inside shapefile dbf file led to really bad result (all non-ascii characters erroneous)

Is there any other parameter, settings or something inside python/arcgis for server/windows what should i try to change to get it work ? the best would be if I could import utf-8 shapefiles, but solution for particular codepage (1250) will be also ok :)


Radek Pohl