Latest Contributions by larryzhang

‎03-26-2011

Just be informed that we have implemented a cluster server infrastructure (a little different from conventional cluster server, which is mainly focusing on 'Ethernet SAN Drives share/transfer' between two nodes after failover) for ArcGIS Image Server 10 to support Mosaic Dataset model and manage massive SAN storages as well. So, our ArcGIS Image Server 10 at cluster can manage ArcGIS Server 10 with Image extension to support load balancing & failover, even though ArcGIS Server does not automatically support the cluster installation. In other word, as long as massive SAN storages at cluster resources can be automatically transfered via Cluster Manager from one active node to another after failover, ArcGIS Server with Image extension can be installed easily just like standalone installation in Windows Server 2008. During the configuration, however, we should pay attention to the following: Within Failover Cluster Manager in either node, both ArcGIS Server Object Manager & ArcGIS SOC Monitor must be managed into Other Resources as well. In this way, the system will ensure all image services working fine after failover, when clients access via a virtual cluster host (IP). Otherwise, all those image services in one node (which are visible) will be invisible in another node after failover, except that ArcGIS SOC Monitor is manually re-started. ++++++++++++ SAN storages look just like local hard disks (Drive N, Drive M, ...), which are managed as cluster resources in cluster server. Link to http://en.wikipedia.org/wiki/Storage_area_network for the details. NAS storages shared over network onto nodes without managing them as cluster resources in cluster server. So far, on our tests, with NAS storage, too slow & poor performance are showed in the following two cases: 1. creation of mosaic dataset (with local datum) from raw bundle of images (with UTM/WGS84) like GeoEye-1, IKONOS, QuickBird, etc; 2. access of image services (published from mosaic datasets with overviews), in particular, one mosaic dataset with thousands of vertices in footprint.

‎03-17-2011

Please install SP1 (ArcGIS 10), which was released on Nov/Dec 2010. The issue �??some of footprint operations are not functional�?? was addressed within this Service Pack 1. At least, it includes: NIM059705 - Cannot shrink footprints when trying to reset footprints of a mosaic dataset. NIM059867 - When re-computing footprints for a table-based raster type containing ortho in the function chain, the footprints disappear. NIM059869 - When running the build footprints tools on imagery containing a range of pixel values to be ignored (typically? �??compressed imagery), the output footprints generated are self intersecting and invalidates the MD, causing all other operations after the build footprint to fail (including build boundary). Please link to http://downloads2.esri.com/support/downloads/other_/ArcGIS-10sp1-issues.htm for more.

‎03-11-2011

Generally, mojarity of those script tools are located at Raster/Mosaic Dataset toolset in ArcCatalog 10. On my practices, if working on plenty of images (with enough overlapping areas, if using shrinking parameter) to remove areas of Nodata, the following steps should be used: Step 1: to change the footprint setting on the Mosaic Dataset Properties dialog box on the MosaicDataset. Click the Defaults tab and change the Always Clip the Image to its Footprint property to Yes; Step 2: to use the standard ArcMap feature editing tools to do manual editing operation. or Step 3: NoData values can also be automatically removed from display in a mosaic dataset using the Define Mosaic Dataset NoData tool. This tool will allow us to specify more than one NoData value; (Just be careful, when you are using the shrinking parameter, in particular, if the images with limited overlapping areas). Please link to http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Define_Mosaic_Dataset_NoData/00170000008q000000/ for the details; Step 4: to re-define or modify the footprints using the Build Footprints tool to perform some systematic calculations, please refer to the link http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Recalculating_footprints_radiometrically/009t0000003w000000/ for more understanding; If you are following the above-mentioned steps, your results of MosaicDataset (with or without overviews) should be reproducable. In fact, it also can be used to clip off the collars of scanned maps or remove the black frame and fiducial marks around georeferenced, scanned aerial photography, especially, with manual editing operations. Besides, on my experiences, those script tools in ArcGIS 10 are much more flexible, efficient, and accurate than similar ones in ArcGIS 9.3/9.3.1 (Image Service Editor)

‎02-19-2011

In our practices with raw bundle of GeoEye-1, we found that the pan-sharpening transformation method may bring too much histogram variance, which causes significantly-unbalanced color in overall image among imagery scenes in mosaic dataset (natural color band 321 with near-infrared weight value added). So far, we have tested 3 isolated areas, i.e., 30-40 scenes/per area, totally, 125 scenes (roughly, 11,000 sqr km), within either one single mosaic dataset or seperate three datasets (http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//0017000000sw000000.htm and http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009t00000041000000.htm). All results are not accepted by our customers. Do we have any other approaches to apply for GeoEye-1 in order to get pan-sharpening transformation done and also can minimize histogram variance for better color-balanced image in overall? Or, on color-balanced aspect, does anyone else have good practice with plenty of images in 'production' test and get acceptable results to share ? +++++++++ In ESRI Image Server 9.3 /10 (image service definition file or mosaic dataset), ESRI algorithm for pan-sharpening is described below: (http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=3263&pid=3249&topicname=Using_the_Pan-sharpen_process 😞 The ESRI pan-sharpening transformation uses weighted averaging (WA) and the additional near-infrared band (optional) to create its pan-sharpened output bands. The weighted average is calculated by using the following formula: WA = (R * RW + G * GW + B * BW + I * IW) / (RW + GW + BW + IW) The result of the weighted average is used to create an adjustment value (ADJ), which is then used in calculating the output values, as shown in the following example: ADJ = pan image - WA Red_out = R + ADJ Green_out = G + ADJ Blue_out = B + ADJ Near_Infrared_out = I + ADJ For the ESRI pan-sharpening transformation, the weight values of 0.166, 0.167, 0.167, 0.5 (R, G, B, I) provide good results when using QuickBird imagery. It has been found that by changing the near-infrared weight value, the green output can be made more or less vibrant.

‎01-28-2011

The error message looks clear. Maybe, what you need to do is to contact your local distributor and get the updated ECP file for your standalone ArcGIS Image Server 10, and then license it. If successful, please register it again! On ESRI advices, however, you may have Image extension installed rather than standalone ArcGIS Image Server 10, after ArcGIS Server 10 is setup, except that you are users of standalone ArcGIS Image Server 9.3/9.3.1. The standalone ArcGIS Image Server 10 is only used for smooth transition (the standalone ArcGIS Image Server 10 is last and final release from ESRI). That is, you can keep using old image service defination files as image services and also publish those files (similar to mosaic datasets) to ArcGIS Server 10 with Image extension as image services. +++++++++++ Personally, it may be good strategy to keep improving this standalone ArcGIS Image Server in order to keep using image service definition files (that is, as a permanent/separate product), at least, in next 3-5 years. For example, to support more infrastructures like cluster server, easily register with ArcGIS Server 9.3/9.3.1 or 10 on Linux & Unix (as separate image extension), easily update, and develop more client add-on applications for oil and engineering applications. If so, ESRI may have more market shares in imagery serving aspect, compared to only selling the integrated ArcGIS Server 10 or higher with image extension because, we (maybe, including non-ESRI users) do see many benefits from standalone ArcGIS Image Server like flexibility and good performance. Of course, other users still may have a choice to use ArcGIS Server 10 with image extension directly for Mosaic Datasets, even, use both standalone ArcGIS Image Server and ArcGIS Server with Image extension for different applications / end users in large organizations.

‎01-11-2011

Hi, team, Any advice? http://forums.arcgis.com/threads/20862-ArcGIS-Server-10-Installation-and-configuration-in-cluster-server?p=67640&viewfull=1#post67640

‎01-04-2011

If you don�??t start with the wizard (i.e., create a fresh image service defination file), you can try �??Maximum data value�?? parameter in the Advanced tab by yourself. The document can be found out at http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Recomputing_footprints_to_remove_unwanted_pixel_data. For some unknown reasons, mostly, it does not work very well for us. It's why manual editing operation is commonly used in our works, when using Image Service Editor.

‎01-03-2011

You can directly use 'Applying the Pan-sharpen process to an image service definition' (http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=3263&pid=3249&topicname=using_the_pan-sharpen_process) for agis server 9.3/9.3.1. It works well for a single xs image (or color-balanced xs mosaic) and single pan image (or color-balanced pan mosaic). However, if planning to use this way for massive unprocessed images, it is not practical because 'color correction' and 'geometric correction' are very limited, or not available at all in Image Service Editor'. Regarding ESRI Image Analysis, two comments are for your reference: �?� it is an efficient and productive application, but only available in ArcGIS 10; �?� the pan-sharpened 'image' (via ESRI Image Analysis) is going to your local temp folder; [Personally, some day, all those derived & temporary 'imagery products' from Image Analysis can be easily transferred and served again for ArcGIS Server 10.] So, if really planning to use this pan-sharpened 'temp image' for ArcGIS Server 9.3/9.3.1, you need efficiently to make it permanent (even 'export' works somehow, but not effectively, mostly not work at all for larger imagery like 100 GB). If succeeded, two ways can be chose to serve for ArcGIS Server 9.3/9.3.1: �?� create an image service definition file with this pan-sharpened image by Image Service Editor and allow agis Server 9.3.1 to access & manage it as an image service; �?� directly serve it as a file (IMG, TIFF) into ArcGIS Server 9.3/9.3.1 From operational perspective, the first one is flexible and manageable, which allows us conveniently to manage plenty of image resources and easily create large image service definition files in data center, mostly apart from the machines hosting agis Server.

‎01-02-2011

On our practices, the image service editor (in desktop ArcGIS 9.3/9.3.1 & 10) does not provide any 'very effective' configuration or automatic functions to help us re-define footprints on user's needs, for example, to remove 255 255 255 or any artifacts along imagery boundary, which may affect interpretation after merging. So, after finishing the wizard, the first thing to do is: manually to edit individual footprint one by one until satisfied, and save your modified footprints. And then, at least, the following steps should be used: �?� re-build image service boundary; �?� remove the previous overviews; �?� re-define overviews; �?� ... and finally, �?� re-generate derived images & re-compile image service If having ArcGIS 10 installed, you can have many automatic functions to do so in Mosaic Dataset. However, it is worth to mention that, very often you are still required to conduct manually-editing operation on some of footprints or boundary, even though the automation can remove majority of artifacts in ArcGIS 10.

‎12-28-2010

We are also in learning curve, but would like to share our practice with you. First of all, as necessary steps for pansharpening individually or as a batch, 'Add Rasters' & 'Raster Properties' configuration will be involved . You can refer to the document at http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009t0000003r000000.htm and http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Raster_type_properties/009t0000003t000000/ Secondly, with GeoEye sources, one snapshot of 'Raster Properties' configurations (including pansharpen options) is attached as an example, which is different from the default. With this configuration, we basically mimic the current imagery processing workflow (like natural color RGB), and will also have operational flexibility in next several steps to enhance mosaic dataset; for example, to make use of the existing or historical 'manually-processed 3-band reference images' as color target during 'color correction'. In fact, it will also minimize many imagery histogram variations in very-high-resolution images (no stretches at all) during building histogram statistics. Honestly, we are still exploring how to fully use the 4th near-infra band, so that we can enhance vegetations but still be able to ensure no more image histogram variations created. For sure, we also would like to have ESRI advice on above configuration, especially, band weights, when choosing bands combination 321 in 'Sharpen' or 'Sharpen More' mode. ++++++++++++++++++++++++ PS., Owing to huge histogram variations in very-high-resolution images, it is very hard for the computer system to conduct completely automatic color correction, in order to prduce good color-balanced mosaic in practice. Also, it is not flexible to meet different customers�?? subjective needs in color, if only using automatic color correction. So, in addition to 'color target' algorithm in 'color correction' tool, a more efficient and flexible tool like �??histogram adjustment�?? application may be required somewhere, in order to manually adjust individual image histogram to match the overall and desired color (after automatic color correction and before finalizing mosiac dataset.)

‎12-27-2010

Hello, team, With our customized PRJ (Atatched .zip, saying our own 'LCC' or 'Ain el Abd 1970 Aramco Lambert 2'), building statistics on mosaic datasets couldn't be successful. Basically, its operation took a week not to finish. However, building statistics on same mosaic dataset with 'WGS84 /geodetic or UTM' can be successfully accomplished in one day (without ArcGIS 10 SP1) or hours (with SP1). The sources are GeoEye (UTM/WGS84). During Add Rasters, the transformation is (Ain_El_Abd_To_WGS_1984_2). In fact, with 'International 1924 /Ain el Abd 1970' in ArcGIS 10, we have other three problems, which were posted at http://forums.arcgis.com/threads/19832-Spheroid-Name-International-1924-in-ArcGIS-10?p=63756&viewfull=1#post63756. Those are related to raster images in IMG and TIFF. Look forward to hearing from you.

‎12-19-2010

Hi, Team, On color correction, we are testing in one area with our own data of three sensors, where it is located at the range (E 50 00'-50 15' / N 26 00' - 27 00'). As a good benchmark from ESRI, we use the very-high-resolution image in Basemap (World Imagery) within ArcGIS Explorer 1500 from USGS as a reference point, which presents well-balanced color in this area. And also it gives well spatially-matched imagery mosaic on overlapping areas (open the attached *NMC AOI file). During tests, what we are trying to do was to create three mosaic datasets with IKONOS (11 tiles), QuickBird (30 tiles), and GeoEye (48 tiles), which cover same area. The target is that we would have similar look and internally spatial-match mosaic image that ArcGIS Explorer 1500 demonstrates. Very regrettably, after many trials with different combination of paramenters in 'color correction' tool (http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009t00000041000000.htm), we were not able to get similar 'color balanced' mosaic dataset from either IKONOS, QuickBird, or GeoEye. Mostly, each scence from 40 has different color (see the attached graphics, which is the best two), and also shows un-clear features (compared to USGS image or our own processed image by ERDAS). We are not sure if having done something wrong so that we couldn't reach this goal. It is worth to mention, during our tests, the 'spatial shifts' also couldn't be solved with only using sensor model RPC and DEM by ArcGIS 10. As a small example, the shifts in the overlapping images between different sensors in that area exist. Also, inside this mosaic image, it has significant internal shifts, corresponding to our surveying data (roads, buildings, etc). The offsets with 3-8 m are very common for GeoEye. (see the attached. You can assume, the background Landsat was spatially rectified, which means that the high-resolution image is required to correct in ArcGIS 10, in order to match the background image or surveying data. For more, link to another two posts at http://forums.arcgis.com/threads/16460-Image-Analysis-in-ArcGIS-10?p=62028&viewfull=1#post62028 and http://forums.arcgis.com/threads/18181-spatial-rectification-(geometric-correction)-in-ArcGIS-10?p=57657&viewfull=1#post57657 Look forward to hearing from you! [ATTACH]3939[/ATTACH]

‎12-17-2010

For Image Classification in ArcGIS 10, as easy and effective as Image Analysis, as long as the working area (AOI) is not too large in each operation. (We have tested 15,000 sqr km of imagery, including IKONOS, QuickBird, and GeoEye, even some airphotos for our land managment and encroachments detection). In fact, those two workflows are reliable, stable and flexible for analysis and further operations like class merging and editing. And it is also straightforward for non-imagery endusers like cartographers and geologists to use, especially, able directly to work on image services (ArcGIS Image Server 9.3/9.3.1 and 10) and make use of ground truths (like geological surface map, etc). (http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//00nv00000002000000.htm). Of course, for some unknown reasons, it is regretful that we couldn't directly work on raster catalogs from ArcSDE, including 'clip', etc in PCs. (For raster catalogs, is it a privilege issue controled within ArcSDE? Or, are those functions not available at all?) Really wish that basic accuracy assessment and some advanced imagery segmentation & grouping functions would be available in image classification. If so, it would be useful for imagery professionals as well.

‎12-14-2010

Really want to thank SP1 for ArCGIS 10. Those really help us to incease productivity. With just released SP1 for ArcGIS 10, it is amazing for some good improvements, especially, efficiency and stability, within image analysis and building pyramids on plenty of raw imagery (workspace). In addition, some functions such as 'composite' and 'difference' also can be directly working on larger image service(s) without 'clip' first. Certainly, very fast and productive! On Mosaic Dataset, however, it is still lacking of effectiveness to do statistics, so that it is really hard to quickly adjust color in the mosaic image, especially, when using the color of reference image (manually-adjusted image) for re-tuning. From initial results that we got so far, it looks not very promising. Anyhow, we are still testing ... Spatially, it looks that GCPs (ground control points) of the references are still not considered to re-adjust Mosaic Dataset, while or after 'Geometric Function' is used to remove distortions. In fact, based on our tests, with IKONOS, QuickBird, and GeoEye images, the current function/chain still produce 5-15 pixels shifts, even though high-resolution DEM is used, in particular, densely-populated places or overlapping images acquired on different dates. For GeoEye, 3-10 m shifts are very common, if only RPC & DEM used (without combining surveying and historical spatial references). Really wish that collecting GCP from references can be available somewhere on some of images and then those GCPs can be used to spatially re-tune mosaic dataset.

‎12-12-2010

Dear ESRI friends and others: We need your feedback and prompt assistance on the following issue: standalone ArcGIS Image Server 10 at cluster server couldn't communicate with remote ArcGIS Server (9.3/ 9.3.1/10); and ArcCatalog 10 couldn't manage GIS services of remote ArcGIS Server 10 (Linux). Question description: A cluster server (Windows Server 2008 R2) is built, which is consisted of two physical servers (active and passive) using a virtual host name (saying IT-IS100). This virtual and clustered server shares cluster resources like Drive M, N, K, or many SAN storages. Both ArcGIS 10 and Standalone ArcGIS Image Server 10 are installed in one of clustered resources (saying Drive M). Standalone ArcGIS Image Server can be started with this virtual & clustered host name IT-IS100 (using default port: 3982 and 3983). Through this virtual & cluster host name IT-IS100, we remotely used the Image Service Editor in ArcGIS 10 to create some image services, also build some mosaic datasets with ArcGIS 10 ... Also, we successfully published all those image services with standalone ArcGIS Image Server 10 at cluster server; then they are applied on security (image services) and assigned onto remote users to connect. Users and remote desktop ArcGIS (9.3/ 9.3.1/ 10) can access those image services... The failover also works, when one of physical server is down, and another up. In other word, always one server is active with the virtual name IT-IS100. So far, all look fine and work well! However, when we recently take the following further actions, no lucks at all: 1. when remote ArcGIS Server 9.3/9.3.1 (UNIX, Windows) tried to register with standalone ArcGIS Image Server 10 sitting at this virtual & cluster server, it failed. (http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html?TopicName=What_is_raster_data%3f#/ArcGIS_Server_Image_extension_version_10_versus_9_3/009t000001vn000000/). 2. when remote ArcGIS Server 10 with Image extension (Linux) tried to add or register with this standalone ArcGIS Image Server 10, it fails. 3. when we tried to publish mosaic datasets from this new cluster server to remote ArcGIS Server 10 with image extension (Linux), it fails. In fact, we couldn't add remote ArcGIS Server 10 within ArcCatalog to manage services. [Please note that we can add remote ArcGIS Server 10 from this cluster server to use GIS services. ] With aids from IT guys, we found out that the port 3982 is not listening at all within this cluster server , but the port 3983 listening. IT fellows did some checks, including firewall (off), opening the port 3982 to all applications, etc. After many tries, finally, they have a feeling that standalone ArcGIS Image Server 10 may have some unknown issues in cluster server environment, which cause this kind of communication problem between cluster server and remote ArcGIS Server ... [For sure, it should be responding, when either Inbound or Outbound stream is trying to communicate with this server. In fact, in our current standalone ArcGIS Image Server 9.3/9.3.1 in Non-cluster server, the port 3982 is working fine, so that ArcGIS Server 9.3/9.3.1 can register /unregister with it easily.] Questions: 1. Any advices on installation and configuration of both ArcGIS 10 and standalone ArcGIS Image Server 10 in a cluster server, in order to support both remote ArcGIS Server 9.3/9.3.1 and ArcGIS Server 10 with Image extension? 2. Any advices on installation and configuration of ArcGIS Server 10 with Image extension at this cluster server, in order to directly publish mosaic datsets onto local ArcGIS Server as image services?

Online Status	Offline
Date Last Visited	‎11-11-2020 02:23 AM

My Ideas

Latest Contributions by larryzhang

Installation and configuration Issues in cluster Server

Re: footprint operation in ArcGIS 10

Re: footprint operation in ArcGIS 10

Pan-sharpening algorithm causes unbalanced-color in overall image (GeoEye-1)

Re: Image Server Installation Issue

Installation and configuration Issues in cluster Server

Re: Turn values 255,255,255 into nodata; in image server

Re: How to create image service that can be pansharpened

Re: Turn values 255,255,255 into nodata; in image server

Re: How to create image service that can be pansharpened

Fail to build statistics on Mosaic Dataset with customized PRJ

correction on color and spatial shift at ArcGIS 10

Image Classification and Image Analysis in ArcGIS 10

Image Analysis and Mosaic Dataset in ArcGIS 10 with SP1

Installation and configuration Issues in cluster Windows Server 2008

Re: GIS to SAP Integration

Re: Open/ Use Lpkx ( new layer package )

Re: How to calculate shortest path?

Re: Cokriging with a categorical covariable?

Re: Cokriging with a categorical covariable?

Python Snippets

R-ArcGIS

ArcAPI

GIS for Geology

Web AppBuilder Custom Widgets