Latest Contributions by Jean-BernardGariépy

‎11-26-2019

I was using python v3.7 Using python v3.6.8 works fine

‎11-25-2019

Hi! I'm batch geocoding millions of adresses using our dedicated Enterprise ArcGIS Server. Some of the batch geocoding results end up being pretty bad, so we geocode them again one by one using the same geocoding service... and usually get much better results than batch geocoding (but that's a whole different issue). This sample code is just to prove the point. Every single call to the geocode() function make my python process add an extra 3MB of RAM. Eventually it reaches 40GIG+ of RAM. That is not suitable for a production environment. But sys.getsizeof(geocode) doesn't change. Any clue? I've implemented 2 different ways and both do the same. By the way, the batch geocoder also have a memory leak, but much smaller. from arcgis.geocoding import geocode from arcgis.gis import GIS import sys gis = GIS('https://our_server/portal') while True: single_result = geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0] print(sys.getsizeof(geocode)) # from arcgis.gis import GIS # from arcgis.geocoding import Geocoder, get_geocoders, geocode # from time import time # import datetime # import sys # geocoderlist = get_geocoders(GIS()) # geocoder = Geocoder(geocoderlist[0].url) # #geocoderlist = get_geocoders(GIS('https://our_server/portal')) # #geocoder = Geocoder(geocoderlist[0].url) # while True: # single_result = geocoder._geocode('3623 Rue Saint-Denis, Montréal, QC H2X 3L6')[0] # print(sys.getsizeof(geocoder))

‎02-19-2013

I just removed ST_TRANSFORM and the query runs really fast ! The problem is behind ST_TRANSFORM. Perhaps its related to my previous post on ST_TRANSFORM as well !? http://forums.arcgis.com/threads/77998-SDE.ST_TRANSFORM-ERROR?p=274138#post274138 create table ST_GC_SINGLEHOUSE as select a.idadr as featureid, a.nocivq as housenumb, trim(o.odogener || ' ' || o.odoparti) as prefixtype, trim(o.odospeci) as streetname, trim(o.odoorien) as suffixdir, trim(m.munnom) as city, CP.CODPOS as ZIPCODE, sde.st_x(a.SHAPE) as LON, sde.st_Y(a.shape) as lat, a.seqodo as streetid, a.shape as shape from st_adresses a, aq_odonymes o, aq_municipalites m, aq_cp_adresses cp where a.seqodo = o.seqodo and a.codemun = m.codemun and a.idadr = cp.idadr;

‎02-19-2013

Good afternoon everyone! I�??ve been trying to run the following query on 3.8 million points for a week now. I ran it 4 times with always the same result: it runs for 8+ hours and then SQLPlus idles for ever and SQL Developer tells me the query ended with an error. STATUS Duration SQL_ID Session_id Session_serial instance_dop CPU_Time IO_Time Start_Time DONE (ERROR) 67188 099q07k78mx7f 12 12535 1|0 3474.06 2.75 15-FEB-2013 11:06:15 create table ST_GC_SINGLEHOUSE as select a.idadr as featureid, a.nocivq as housenumb, trim(o.odogener || ' ' || o.odoparti) as prefixtype, trim(o.odospeci) as streetname, trim(o.odoorien) as suffixdir, trim(m.munnom) as city, CP.CODPOS as ZIPCODE, sde.st_x(sde.st_transform(a.SHAPE,4269)) as LON, --epsg 4269 : NAD83 sde.st_Y(sde.st_transform(a.shape,4269)) as lat, a.seqodo as streetid, a.shape as shape from st_adresses a, aq_odonymes o, aq_municipalites m, aq_cp_adresses cp where a.seqodo = o.seqodo and a.codemun = m.codemun and a.idadr = cp.idadr; If you wonder why I'm using EPSG 4269 here see my previous post: http://forums.arcgis.com/threads/77998-SDE.ST_TRANSFORM-ERROR?p=274138#post274138 In the above query, ST_Adresses in stored in ESRI ST_GEOMETRY. So I ran the same query on the same 3.8 millions points but this time using SDO_Adresses stored as SDO_GEOMETRY Oracle Spatial format. All the other tables are the same and indexes are built on each single join field in both cases on top of having spatial indexes. So the same query using SDO_GEOMETRY completes successfully in 42 min 40 sec. create table SDO_GC_SINGLEHOUSE as select a.idadr as featureid, a.nocivq as housenumb, trim(o.odogener || ' ' || o.odoparti) as prefixtype, trim(o.odospeci) as streetname, trim(o.odoorien) as suffixdir, trim(m.munnom) as city, CP.CODPOS as ZIPCODE, SDO_CS.TRANSFORM(A.SHAPE,4326).SDO_POINT.X as LON, SDO_CS.TRANSFORM(A.SHAPE,4326).SDO_POINT.Y as lat, a.seqodo as streetid, a.shape as shape from sdo_adresses a, aq_odonymes o, aq_municipalites m, aq_cp_adresses cp where a.seqodo = o.seqodo and a.codemun = m.codemun and a.idadr = cp.idadr; What�??s wrong with my ST_GEOMETRY query ?

‎02-19-2013

I get the following error when trying to transform points stored in ST_GEOMETRY EPSG 32198 into WGS 84 EPSG 4326: ORA-20603: Spatial References are not compatible. ORA-06512: at "SDE.ST_GEOMETRY_SHAPELIB_PKG", line 720 ORA-06512: at "SDE.ST_GEOMETRY_OPERATORS", line 2623 All our data is stored in EPSG 32198 and according to spatialreference.org (http://spatialreference.org/ref/epsg/32198/prettywkt/) this projection is based on datum NAD83 ESPG 4269. It looks like ST_TRANSFORM is having a hard time to convert NAD 83 EPSG 4269 based projection (like 32198) into WGS 84 EPSG 4326. Which I believe to be a very simple task for any GIS software. The conversion between them, at least in North America should be straight forward. Weird isn't it? The work around I found was to use NAD 83 EPSG 4269. sde.st_x(sde.st_transform(a.SHAPE,4269)) sde.st_Y(sde.st_transform(a.shape,4269)) And it worked! Hope this can help someone!

‎01-23-2013

Good morning, I'm running an 10.1 enterprise geodatabase on Oracle 11g R2 and my 10.1 desktop runs on XP. Using the Oracle 10g client I get the following bug when I try to copy data into the Oracle geodatabase: ERROR 999999: Error executing function. Insufficient permissions [Error executing stored procedure sde.registry_util.delete_registration::ORA-20025: Not owner of registration 308.] 308 is the SDE.TABLE_REGISTRY.REGISTRATION_ID number used by ArcGIS. After installing the 32 bit version of Oracle 11g Client... everything is back to normal. Cheers! jb

‎12-14-2012

ESRI has been assisting me with this bug and I just got an answer from ESRI. I looks like I've put the finger on the problem with my regional setting bug... The work around would be to change the regional settings to English! Regarding the Oracle bug, it actually looks like Oracle has not adequately updated their bug database. Upon further investigation on our bug (NIM069549) that references the Oracle bug, there is an activity logged by one of the Esri Inc support staff. He mentions that he spoke to Oracle in May 2012, and that Oracle has actually closed this Oracle incident (12537431) as NOT a bug. This analyst says that the reason given is because OCIEnvNlsCreate uses the national language setting of the computer (Nls = National Language Setting). OCIEnvNlsCreate is used when creating an sdo_georaster and the failure occurs when the Oracle xml document is created. Basically ??? the reason that users can experience this Oracle bug is because of regional language settings in either the O/S or Oracle. We have spoken to Esri Inc and have requested that the following be updated to reflect this finding from Oracle: NIM069549 - http://support.esri.com/en/bugs/nimbus/TklNMDY5NTQ5 NOTE: have requested that the alternate solution be changed to state that it can be caused by regional language settings. Also, reason for rejection needs to be changed accordingly Error: An Oracle LPX-0019 error occurs when inserting data into an SDO_GEORASTER field -http://support.esri.com/en/knowledgebase/techarticles/detail/38940 NOTE: have requested that the solution/workaround be changed to state that it can be caused by regional language settings. Suggested to try changing to English language setting. ArcGIS and the Oracle Spatial raster type - http://resources.arcgis.com/en/help/main/10.1/index.html#//002n0000006z000000 NOTE: this is the article with the known limitations at the bottom. Since metadata is in XML format, it is quite possible that the commas you were getting with the French language setting were directly related to this issue. The Oracle XmlLoadDom function returns the erorr, so you will see this error in Oracle, not in ArcMap.

‎12-13-2012

Thank 😉 Now the issue is the accuracy of article #38940. http://support.esri.com/en/knowledgebase/techarticles/detail/38940 We never experienced the bug mentionned in that article! Also, in the "Known limits of using SDO_GEORASTER with a geodatabase" section of the following ArcGIS resources page http://resources.arcgis.com/en/help/main/10.1/index.html#//002n0000006z000000 I am not sure if it is appropriate to say that "You cannot use SDO_GEORASTER storage in an Oracle 11g R2 database as a result of Oracle bug 12537431." Sure enough oracle bug #12537431 - XMLLOADDOM FAILS WITH LPX-00019 WHEN SESSION IS CREATED WITH OCIENVNLSCREATE is still active on Oracle's Metalink website, but on our side it does not prevent us from using SDO_GEORASTER. My original post was asking if anyone ever experienced the problem mentionned in Article 38940. On the web it looks like I'm the only one that tried to reprocude the bug which we where not to do in ArcGIS 10.1. Does anyone on the Internet ever experience that bug with SDO_GEORASTER in Oracle 11g R2 ??

‎12-12-2012

I've solved the problems. We are now able to insert data into SDO_GEORASTER without any problem. We've never encountered the following but as describe by ESRI : http://support.esri.com/en/knowledgebase/techarticles/detail/38940 The first problem was that our Oracle SDE Schema was originally created with ArcGIS 10.0. So we've setup a new database with 10.1 SDE Schema. Second, the regionnal settings of my ArcGIS Desktop computer was French Canadian. So ArcGIS was inserting comas as decimal separators in the raster metadata. I've found this by using the following to SQL commands: SDO_GEOR.validateGeoRaster() SDO_GEOR.schemaValidate() We are now able to insert data into SDO_GEORASTER using the following geoprocessing tool: Data Management Tools => Raster => Raster Dataset => Copy Raster Now we're having other issues bulding the pyramids using the SQL statement SDO_GEOR.generatePyramid() because ArcGIS doesn't assign any SRID to the raster 😕 I will start a new thread for this bug soon as I trouble shoot a bit more.

‎11-26-2012

Good news !! With the help of some skilled friends we manage to come up with something that works just perfectly !! And this, without any alias list !!! <section desc="Zones"> ... <def name="GenPostal"> <alt ref="FSALDU" /> <alt> <elt ref="Fsa"/> <elt ref="OptLdu"/> </alt> </def> <def name="Fsa"> <alt>`[a-zA-Z][0-9][a-zA-Z]`</alt> </def> <def name="Ldu"> <alt>`[0-9][a-zA-Z][0-9]`</alt> <alt>`[0-9][a-zA-Z]`</alt> <alt>`[0-9]`</alt> </def> <def name="FSALDU"> <alt> <elt ref="Fsa" weight="100"/> <elt ref="OptHyphen" weight="0"/> <elt ref="Ldu" weight="50" pre_separator="optional"/> </alt> </def> <def name="OptLdu"> <alt/> <alt fallback="true"> <elt ref="OptHyphen" weight="0"/> <elt ref="Ldu" weight="50" pre_separator="optional"/> </alt> <alt fallback="true"> <elt ref="word" pre_separator="none"/> </alt> </def> </section> <section desc="ZonesNoSearch"> ... <def name="OptPostalNoSearch"> <alt> <elt ref="GenPostalNoSearch"/> </alt> <alt/> </def> <def name="GenPostalNoSearch"> <alt> <elt ref="FSANoSearch" match_as="Fsa"/> <elt ref="OptLdu"/> </alt> <alt fallback="true"> <elt ref="FSANoSearch" match_as="Fsa" /> <elt ref="word" pre_separator="none"/> </alt> </def> <def name="FSANoSearch"> <alt>`[a-zA-Z][0-9][a-zA-Z]`</alt> </def> </section>

‎11-26-2012

Don't know if it is in anyway related nor of any help, but I saw this old thread on an OTN forum related to the same "OCIEnvNlsCreate" function and issues with it in PHP: https://forums.oracle.com/forums/thread.jspa?threadID=486961 National Language Support do seems to be the issue as also reported by the Metalink Workaround posted earlier. If a simple environment variable could solve the problem!!! that would be amazing !!

‎11-26-2012

ESRI more or less already did that, see the following links: ArcSDE SDK 10 C and Java API and ArcGIS Spatial Data Server REST API The tests are made using the Dirrect Connect with Oracle 11g client installed on the client computer. Those API are great at what they do, but as for using them to manage data in a RDBMS sounds like taking an airplane when you can actually bike to work 😉

‎11-26-2012

This time I got this Error popup !! [ATTACH=CONFIG]19527[/ATTACH]

‎11-26-2012

The copy and paste is probably mostly a server task. Unfortunatly, to copy/paste a raster within the same Database ArcCatalog needs to read the file through the network on the client machine and send it back to the server. I agree that it would be much faster if ArcCatalog could acheive the same task only on the server. Even better, it would be nice if ESRI could release an SQL API to manage spatial features from the RDBMS. But I'm going away from the question here... To copy the same raster from BLOB into a BLOB with a different name, it takes about 1 minute. Also, by looking at the network card activity you can clearly see that ArcCatalog is downloading the raster from the server and pushing it back. Also, you can look at your CPU activity and it clearly tells you that ArcCatalog is working on it. When copying from BLOB to SDO_Geometry, for 1h+, ArcCatalog never even tried to download the file from the server (no network activity what so ever) and no CPU activity. It clearly fails at the task, without any error message. On the Oracle Server, my DBA tells me that my session is open, but inactive. I'll ask him to check the logs to see if there is any error message generated. Finally, there is a Workaround on MetaLink which we cannot apply ourself... maybe ESRI could look at it ? XmlLoadDom fails to parse and load an XML document when the OCI session is created with OCIEnvNlsCreate function. But the same works fine when session is created with OCIEnvCreate function.

‎11-23-2012

Hi! Is there anyone that was able to reproduce the following bug ? http://support.esri.com/en/knowledgebase/techarticles/detail/38940 I tried using a 10.0 ArcGIS Desktop Client and a 10.1 ArcGIS Desktop Client on a Oracle 11g R2 with all the appropriate keywords set in DBTUNE. A simple copy/paste of a raster stored as a BLOB into an SDO_GeoRaster type make ArcCatalogue to freeze indefinitely (more than 1h) doing nothing. No error message. Also, I haven't found any post on the web regarding this bug. We don't mind calling Oracle for this bug... if we can manage to reproduce it. Any clue on how to reproduce this bug would be greatly apprciated.

Online Status	Offline
Date Last Visited	‎11-11-2020 02:23 AM

My Ideas

Latest Contributions by Jean-BernardGariépy

Re: python geocoder api memory leak

python geocoder api memory leak

Re: ST_Geometry long operation error

ST_Geometry long operation error

SDE.ST_TRANSFORM ERROR

10.1 / Oracle "Insufficient permissions ... ORA-20025: Not owner of registration" Bug

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: Geocoding Postcode with space (Canada)

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: SDO_GEORASTER BUG on Oracle 11g R2

Re: SDO_GEORASTER BUG on Oracle 11g R2

SDO_GEORASTER BUG on Oracle 11g R2