Writing to CSV issues
Dan
Attached is a screenshot of my GEE Service. My input detects a change of "status" on a Sewer overflow, if that status is "Verified" it strips out a bunch of unneeded data and pushes the remainder to a CSV file.
I have a python script that uses that CSV file to suck in the info and do some processing. I need that CSV file to always be named the same thing, and to only contain the value not the fieldname & value (in seperated fields) as it does now.
Thanks
Here is an answer from RJ Sunderman which indicates, sadly, I can't do what I want unless I hardcode the Filename into my Python script:
Hello Andrew –
I think there is some confusion in how the ‘Output Date Format’ and ‘Custom Date Format’ properties of a ‘Write to a CSV File’ output are used … and from where the “FieldName” to which you are referring is coming.
First, the ‘Write to a CSV File’ output uses the out-of-the-box Text adapter. This adapter’s implementation has some particular characteristics which a lot of folks find useful. The “FieldName” you are referring to, I think, is actually the name of the GeoEvent Definition associated with the GeoEvent delivered to the output.
For example, I have a GeoEvent Service which accepts event data sent to a GeoEvent hosted REST endpoint via HTTP/POST. The event data sent looks like this:
[
{
"TrackID": "AA-1234",
"TimeStamp": "3/15/2015 12:34:56",
"SensorCode": 11210
}
]
The GeoEvent Definition I created for my input to use, named GenericJsonReceiver, looks like this:
When I configure a ‘Write to a CSV File’ output I can leave the default ‘Output Date Format’ setting (‘ISO 8601 Format’) and I’ll get output in my named CSV file which looks like this:
GenericJsonReceiver,AA-1234,2015-03-15T12:34:56.000-07:00,11210
There are not field names in the data string. There is only the three data values prefixed with the name of the GeoEvent Definition. You cannot turn off this behavior. One of the features of the Text adapter is that it always prepends the name of the GeoEvent Definition associated with the event data. Precisely because you don’t have field names, seeing the name of the event definition can be very helpful. Also, this is one of the only outbound connectors you can use when you’re unsure which GeoEvent Definition is associated with your data once it is through the filtering and processing of a GeoEvent Service … and you want to find out.
If I change my ‘Write to a CSV File’ output configuration to specify a date/time mask, I can control what date/time value look like (e.g. how they are represented as String values) in the data written to the file. If I specify the ‘Output Date Format’ and ‘Custom Date Format’ properties as follows:
Then the text logged into the CSV file will appear as:
GenericJsonReceiver,AA-1234,15-Mar-2015 12:34:56,11210
What I’ve done is tell the output that I would like to see the date formatted as days, followed by a three letter abbreviation of the month, and a four digit year, then the time. I’ve also specified that I like dashed used in my date rather than slashes. These settings have absolutely nothing to do with the name of the output file.
The name of the output file is static, and configured up above the ‘Advanced’ parameters. You only get to specify a file prefix. Depending on how you elected to have the file rollover (the ‘File Rollover Method’ property) you will either get your specified file prefix with a .csv suffix … or you will get a system generated date/time stamp following your specified prefix with a .csv suffix. You have no control over how the transport (the out-of-the-box File transport in this case) decides to represent the date/time it integrates into the output file’s name.
Hope this information helps –
RJ
