ArcMap can't read string in TXT file

YichiLiu1 · ‎12-13-2013

Hi,

I have a bunch of txt files that I need to convert to shapefiles. I first wrote a simple script to format the data as comma delimited. But when I opened the converted files in ArcMap, some of characters didn't show up. For example, "COUSOU" would show up as <NULL>, "88F6" would show as "886". But, in some files, they worked fun. Any one knows why and how to deal with it?

Here is my code for converting file. Any though would help me a lot. Thank you very much!

inFile = open (arcpy.GetParameterAsText(0),'r') outFile = open (arcpy.GetParameterAsText(1),'w')  buffer = []  for line in inFile::     line = line.strip()     buffer = line.split(' ')     buffer = filter (None, buffer)     outFile.writelines(",".join(buffer))     outFile.writelines("".join('\n'))  inFile.close() outFile.close()

WilliamCraft · ‎12-14-2013

Ok, give this a try in a 10.1 environment. You can run it through Python IDLE or you can double click it to run it through the Windows Command Prompt. Either way, there will be two user inputs it asks for: the first is the full path and file name for the input TXT file; the second is the full path to the directory at which the output files should be written.

[ATTACH=CONFIG]29889[/ATTACH]

Once the script completes, you will see that I decided to actually retain a CSV in addition to the Shapefile you asked for. I did this for a bit of QA so you can make sure what got parsed from the input file matches the Shapefile. Now, here's the thing... your input source files must all be in the same format in terms of field name sequencing. If they differ, then your field names in the Shapefile may not match what the values really represent in the input file. Also, I've not set a spatial reference on your data for the Add XY Event Layer GP tool in the script. You may need to do that for overlay accuracy purposes. Lastly, you're welcome to modify this to create FGDB feature classes rather than Shapefiles. Using a Shapefile means that your field names are limited in character length, so they may look a bit funny.

Let me know what you think! The only thing I'll ask for is, if you are happy with the script and my efforts, please mark this as the correct answer for the thread and then vote up other helpful responses in the thread as well.

View solution in original post

WilliamCraft · ‎12-13-2013

I'm happy to try and help you, but I'll need a bit more info from you first. Please post the input file or at least a few sample lines of what the file contents look like.

YichiLiu1 · ‎12-13-2013

I'm happy to try and help you, but I'll need a bit more info from you first. Please post the input file or at least a few sample lines of what the file contents look like.

Thank you so much! I've attached two text files as examples. I can open test2.txt, however, when I opened test2.txt, "COUSOU" became <NULL> and "88F6" became "886".

YichiLiu1 · ‎12-13-2013

This is test2.txt

WilliamCraft · ‎12-13-2013

I first wrote a simple script to format the data as comma delimited.

Just curious, is the file you provided the modified one from the script or is it the original? Please provide the original if possible.

Which version of ArcGIS are you working with?

YichiLiu1 · ‎12-13-2013

Just curious, is the file you provided the modified one from the script or is it the original? Please provide the original if possible.

Which version of ArcGIS are you working with?

I tried both 10.0 and 10.1. I only have modified data with me right now. I just tried same data on my own computer (I was using computers at work), all of them worked completely fine! I am totally confused. Is is because of the computer settings?

WilliamCraft · ‎12-13-2013

I am not sure of your computer settings, but if you would like me to help with the code please provide an example of the original source file. You said that you were altering the TXT files to become comma-delimited, so whatever you are running the script against is what I would need to start with. It sounds like the script you wrote isn't quite right as it is truncating certain values or you are getting NULL returns on others. It's possible that we could write some alternate Python that would convert your source file(s) directly to Shapefile format without having to produce an intermediate comma-separated file. Without that, I can't troubleshoot your output file.

WilliamCraft · ‎12-13-2013

Alternatively, if you are satisfied with the output from your script then you can use the Add XY Data option in ArcMap to create an XY event layer using the X and Y fields. The rest of the info in your file becomes attribution. Then, right click the layer in ArcMap's table of contents and use the option for Export Data. Pick Shapefile format per your requirements and save the output to a familiar location.

YichiLiu1 · ‎12-14-2013

I am not sure of your computer settings, but if you would like me to help with the code please provide an example of the original source file. You said that you were altering the TXT files to become comma-delimited, so whatever you are running the script against is what I would need to start with. It sounds like the script you wrote isn't quite right as it is truncating certain values or you are getting NULL returns on others. It's possible that we could write some alternate Python that would convert your source file(s) directly to Shapefile format without having to produce an intermediate comma-separated file. Without that, I can't troubleshoot your output file.

Thank you very much! I tried couple times again and figured out that the version of ArcGIS did make a difference. TXT files showed up correctly if I use 10.1. Now, I'm facing a new problem. Some data, for example, "59E13" was read as double instead of string, which exported as 5.900000e+14. Do you know how can I deal with this issue? I would really like to learn how to convert from source files directly to Shapefiles if it will make everything easier. I attached a sample txt file here. I need to extract data from the table at the bottom and convert is to shapefile or feature class. Right now I am using the approach you mentioned, make XY event layer and then export. Again, thank you very much for your help!

WilliamCraft · ‎12-14-2013

Great, thanks for providing the file. Give me a short bit of time and I'll reply with a 10.1 solution for you. I'm going to assume that all of your source files follow this format, too.