[PYTHON] .gdb import - spatial envelope settings not working

[MarekDekys]

Hi,

I'm trying to import .gdb (file geodatabase) file using python

while everything works fine, I'm having trouble defining custom envelope for my import to crop an area out from big .gdb file:

no matter what I put into [] it always loads all 490 objects:

setImportQueryAndEnvelope("OBJECTID < 491", [7157.656, 4162.500,274845.500,5143901.000] ,"NORTH_WEST")

but I'm not sure if the syntax is correct because manual suggests: 

@param env: The spatial envelope with syntax (xMin, xMax, yMin, yMax). (default = None). [sequence of float]

while in dialog there are options for width, height and X-Offset and Y-Offset

I tried the documentation's syntax to achieve 1x1 km crop using  :

[7157.656, 7257.656, 4162.500, 4262.500]

(the 2nd and 4th values are just incremented by 1000 - to correspond with manual's syntax of xMin, xMax, yMin, yMax ) but the output is always the same..

I also tried to put all zeros in dialog window and it correctly loaded 0 objects

but when I try the same thing in code using [0,0,0,0] it still loads all 490 objects and it should load 0 objects instead

any help appreciated. 

thanks,

Mark

[IanMurray]

"@param env: The spatial envelope with syntax (xMin, xMax, yMin, yMax). (default = None). [sequence of float]"

Have you tried entering the values as a tuple instead of a list in the parameters?  Not sure it makes a difference, but the example does show it as a tuple not a list.

[MarekDekys]

Hi Ian, thanks for a quick reply

I tried tuple with zeros using code below (hope it's tuple) to force it not to import anything (to check that it works) but looks like it doesn't work because it still exports everything instead of empty layer with 0 objects

settings.setImportQueryAndEnvelope("OBJECTID > 0", (0.0,0.0,0.0,0.0) ,"NORTH_WEST")

Not sure what they mean by a [sequence of float] - but I found another location in manual where it uses [sequence of float]  and it is defined like a [val,val,val,val] but maybe i'm mistaken:

setAttributeLayerExtents(self, layer, extents)

@param extents: An list of four floating point values. [sequence of float]

ce.setAttributeLayerExtents(l, [0,20,3000,4000])

any help would be appreciated

[IanMurray]

Have you made sure to not only set the envelope and query, but make sure it is being applied in the settings?

Based on your screenshot there is a Boolean to actually use the query and envelope values you put in, which I would imagine equates to getUseSelectionQueryAndSpatialEnvelope() method mentioned in the help.  You are probably writing your envelope parameters correctly, its just not being applied and you need to set that value to True. 

It would be nice if there was better help examples on that documention link for FGDB settings for CE import.

[MarekDekys]

Yes I am using settings.setUseSelectionQueryAndSpatialEnvelope(True). see the code below: (same as in screenshot above) 

the sql selection query works fine (i can set OBJECTID < 101 to import only first 100 objects.. etc..) but it ignores the rest :

from scripting import *

input_location = "_input_buildings_gdb/test_small.gdb"
ce = CE()# get a CityEngine instance

if __name__ == '__main__':
settings = FGDBImportSettings()
settings.setImportAndMapAttributes(True)
settings.setUseSelectionQueryAndSpatialEnvelope(True)
settings.setImportQueryAndEnvelope("OBJECTID > 0", (0.0,0.0,0.0,0.0) ,"NORTH_WEST")
ce.importFile(ce.toFSPath(input_location), settings)

[IanMurray]

Apparently its too early and my reading comprehension is not what it needs to be...

TFuchs-esristaff‌

[DavidWasserman]

Hi Marek, 

On the GIS Side, would it be too much to perhaps just depend on the SQL functionality? I mean if you want to define zones for import, you could just have features within a grid boundary (or census block), be specified by a SQL query? Sometimes I just make the edits on the GIS side to get a smooth pipeline into CE. 

Can you share the exact projection of the data you are importing? 

Also on the python side, I might be missing something, but does the result change if you specify the specific file path/name of the FC you are trying to import using the setFile setting? (I don't use this function a lot, so it might be a silly question). 

setFile(self, stringValue)Sets File field. 
@param stringValue: the new value. [str]

David

[MarekDekys]

Hi David,

If I can use some SQL query to select only objects that have only certain cooridnates (that fit into boundig box) like:

SELECT * WHERE "OBJECTID > 0 AND X > 0 AND X<100 AND Y>0 AND Y<100" that would be great.

but the problem is I don't know if this is possible and the code above doesn't work for me because fields X,Y are not defined as a table (unlike OBJECTID which is defined as a field/table) and there are no fields/tables defined in gdb that contain information about position

I'm using UTM 32N projection

I have also rewritten the code to use getFile/setFile from what I understood from your reply but it works and acts the same as the code before:

from scripting import *

input_location = "_input_buildings_gdb/test_small.gdb"
ce = CE()# get a CityEngine instance

if __name__ == '__main__':
settings = FGDBImportSettings()
settings.setImportAndMapAttributes(True)
settings.setUseSelectionQueryAndSpatialEnvelope(True)
settings.setFile(ce.toFSPath(input_location))
settings.setImportQueryAndEnvelope("OBJECTID > 0" , (0.0,0.0,0.0,0.0) ,"NORTH_WEST")
ce.importFile(settings.getFile(),settings)

If anybody can share any code example about setting spatial envelope via setImportQueryAndEnvelope I would be grateful

[deleted-user-Q-Ku9J1BlYDK]

Hi Marek, 

I was thinking more along the lines of creating a fish net or selecting specifically the data you want to import and adding a field that you can use to flag for it. So you would select the relevant study area in GIS, add an attribute associated with it and then use a SQL query like "INSIDE_STUDY_AREA=1" based on your new field. This is not using the spatial envelope but might be a way to achieve the same thing. You could also use a fish net if you need to import areas in small batches (spatial join the fish net IDs to your  data for example and then select based the fish net ID). 

CE can be odd on some projections. Are you providing spatial envelopes appropriate to your projection? I try to use web mercator (auxiliary sphere) when importing into CE once if the data is not getting imported as I thought it should. It might change the result. 

I thought the setFile/getFile paths need to be the full feature class path. In ArcPy, a specific feature class for example is set up as : "FolderPath/MyFGDB.gdb/SomeFeatureClass". I was curious if maybe this was a necessary setting for spatial envelop but it was just a guess. 

I will try to test this on my end later next week if no one replies Marek. 

David

[MarekDekys]

Hi David, 

thanks for in-depth reply. I agree that a possible workaround would be by adding flags to gdb and then use SQL to cut into regular grid/fishnet but unfortunatelly I don't have access to any good GIS tool from esri.. I only have global mapper which has problems processing .gdb

about setting up correct projection - There is no option in dialog to set the projection nor is in the documentation about 

FGDBImportSettings() so I believe it will reproject it to the current projection of the scene (which works perfectly I have .gdb files in LV95 and it reprojects them correctly into WGS84 UTM 32N) - the problem is that I cannot force it to import 0 objects if I set spatial envelope like (0,0,0,0) to confirm that this command is working - It always imports everything just like it is ignoring this command

not sure what you mean about setFile/getFile. can you provide me with a code sample? (but I think this is not the case ) I don't know is CE uses ArcPy or it is just a regular python..

I have other workaround by processing everything entirely in CE (without need of other GIS tools that I don't have) which I will use if everything else fails but it is a slower proces for milions of files..

I was just hoping that this was only a small issue (in syntax misunderstanding in setImportQueryAndEnvelope command ) that can be solved by somebody providing me a correct code sample but it turns out it won't be so easy as it seems like nobody is using this command.. or it is a bug so I will report it to esri if nobody answers about the correct syntax of this command