Latest Contributions by larryzhang

designO&G downstream large-scale operations drive intelligent infrastructure and assets management throughout the operating life of assets, which significantly impact on safety, operational efficiency, production predictability, and overall profitability. One of the positive effects on asset information management is to seamlessly integrate 3D process plant models, utility 3D models, visualization, simulation and mobility in 3D GIS platform with ‘real-time’ capability. The first step for 3D GIS platform is to build 3D GIS solution database in one eco-system, meanwhile seamlessly interoperating with others. The construction of 3D GIS database are required to integrate different data and data models, in particular, 3D Process Plant Models (refinery, gas plant, petrochemical, power, etc.) and 3D Utility Models (powerlines, pipelines, etc.). 3D Plant Model (as 3D Block in ArcScene) 3D Powerline Model (3D Scene in CityEngine) After reducing the details of 3D plant models by 100-500 times, CityEngine can be used to convert into multipatch via Collada; But, there is no effective way to geolocate the block onto the earth in ArcScene or ArcGlobe in Edit mode 3D powerlines model converted as 3D block in 3D scene However, it is huge challenging in theory and practice to integrate 3D Plant models into geospatial solution platforms like ESRI ArcGIS, because of the different computer precisions between 3D GIS and 3D Plant Models (mostly 3D CAD design models by Microstation and Intergraph in O&G), which demonstrate the following major issues during conversion from 3D CAD to 3D GIS via open standard formats like Collada: 1. too details with huge size (i.e., very common to have 1 - 2 GB or more , if via Collada); 2. non-georeferenced (i.e., most of plant engineers /designers are still not aware of the spatial technology) 3. loss of intelligence (i.e., all attributes/ annotations associated with objects in 3D CAD model are lost in 3D GIS) Worth to mention, asset information models in 3D GIS platform should provide an accurate digital representation of the physical asset (which for many infrastructure assets is in a constant state of flux) and detailed knowledge of the asset context (including the decisions behind why the assets were designed the way they were, as well as how the asset was constructed and modified). Let's compare major capabilities from two products (MicroStation, AutoCAD) for 3D CAD plant modeling (attachment)... Secondly, how to convert the following major 3D process plant models into 3D geospatial solution platforms? Intergraph PDS / MicroStation (v6, 7) MicroStation (v8) / i-models / Hypermodeling / Bentley Map 3D Intergraph Smart 3D / GeoMedia® 3D Still many 3D models in PDS as corporate standand; especially, massive 3D plant models were delivered in PDS direct and seamless integration with 3D Plant & Utility models via i-models; and directly edit over 3D CAD in DWG or DGN, LiDAR point clouds (TranScan) within Bentley Map 3D ... interoperability with both graphics and data attributes of foreign CAD models and PDS models; enabling an even richer, centrally-managed 3D ecosystem, especially, with Bentley Map 3D /i-models So, what is the operational solution to meet those challenges? ... View more

Hi, all When running the sample at ArcGIS Help (10.2, 10.2.1, and 10.2.2) , does anyone have this issue? Using getpass in python console widget echos password in Windows (Python 2.7 or earlier) The code is attached. +++++++++++++++ What steps will reproduce the problem? 1.Open a python interpreter in the console 2.type the following: >>> import getpass >>> notMyPassword = getpass.getpass("type your password: ") password What is the expected output? It shows the prompt "don't type your password: " but the password characters are not echoed to the console window What do you see instead? getpass displays a warning that it couldn't control echo on the terminal, and that password input may be echoed. ... View more

Coastlines, shoals and reefs are some of the most dynamic and constantly changing regions of the globe. Monitoring and measuring these changes is critical to marine navigation and an important tool in understanding our environment. Near shore bathymetry is currently calculated using high-resolution multispectral satellite imagery. However, with the introduction of WorldView-2’s higher resolution, increased agility and Coastal Blue band (400-450 nm), bathymetric measurements will substantially improve both in depth and accuracy, which can be cost-effectively used to apply in operation potentially to replace the tradtional marine surveying, in particular, up to 10 - 20 m in depth. Blue band (396-460 nm) in WorldView-2 & -3 panatrating shallow water (coastal, lake, etc.) * Note: Marine surveyors perform inspections of vessels of all types including oil rigs, ferries, cargo vessels and warships, pleasure craft, passenger vessels, tugboats, barges, dredges, as well as marine cargo, marine engines and facilities such as canals, drydocks, loading docks and more for the purpose of pre-purchase evaluation, insurance eligibility, insurance claim resolution and regulation compliance. There are two established techniques for calculating bathymetry using multispectral satellite imagery: a radiometric approach and a photogrammetric approach. The Radiometric Approach The radiometric approach exploits the fact that different wavelengths of light are attenuated by water to differing degrees, with red light being attenuated much more rapidly than blue light. Analysts have leveraged existing multispectral satellites’ ability to detect light in the blue (450 – 510 nm), green (510 – 580 nm) and red bands (630 – 690 nm) to achieve good depth estimates, in water up to 15 meters in depth. And, with the addition of sonar based ground truth measurements, they have achieved vertical and horizontal accuracies of less than 1 meter. In order to improve bathymetric measurements, analysts have turned to airborne, high-resolution multispectral platforms. These sensors are able to detect light between 400 and 450 nm – the spectrum that provides the deepest penetration of clear water.Studies using these data have shown that accurate bathymetric measurements can be achieved up to 20 meters and deeper. WorldView-2 is the first commercial high-resolution satellite to provide 1.84 m resolution multispectral imagery, plus a Coastal Blue detector focused on the 400 – 450 nm range. WorldView-2’s large single-pass collection capabilities will also make the application of ground truth data more accurate and reliable. Multiple small collections contain differences in sun angle, sea state and other parameters and it is challenging to calibrate one series of measurements and then apply them across a broad area. Large synoptic collections, enabled by WorldView-2’s agility and rapid retargeting capabilities, allow analysts to compare the differing absorption of the Coastal Blue, Blue and Green bands, calibrate their bathymetric estimations using a few known points, and then reliably extend the model across the entire collection area. The Photogrammetric Approach In this method, stereoscopic images are collected over the target area, and a data elevation model (DEM) of the shallow ocean floor is produced from the imagery. Early studies with both satellite imagery, and digital photography appeared promising, and demonstrate that this technique can be used to provide accurate bathymetric models of shallow environments without ground truth. However, the technique has not been widely studied due to limitations in the capabilities of current sensors. The challenge with collecting stereoscopic imagery of the shallow ocean floor is in how light interacts with the air/water interface. At high angles of incidence, light is completely reflected off the surface of the water, preventing any sub-aquatic features from being observed. Current multispectral satellite sensors are not able to collect enough high-resolution stereoscopic imagery within the narrow angle necessary to penetrate the ocean surface. In addition, none of them are able to measure the shorter wavelength blue light necessary for maximum depth penetration. WorldView-2 will make this new method for measuring bathymetry possible. The Coastal Blue band will deliver maximum water penetration, and WorldView-2’s enhanced agility will enable the collection of large amounts of high-resolution intrack stereo imagery at the ideal angle for water penetration. The advantage of this approach is that multiple images can be registered using tie points that are visible on land and in the water, and the resulting stereo composite can be used to calculate water depth without relying on ground truth measurements. No other satellite is able to deliver this unique combination of high spatial and spectral resolution, agility and stereo collection capacity. Please refer to the White Paper from DigitalGlobe in 2010 (attachment), and also the earlier study by Lee (2010). Worth to mention,  Lee 's study didn't showed an obvious correlation between the Coastal depth and Blue bands, because of only using the traditional classification methods in thier study... ... View more

LAS Point Clouds data can be accurate and reliable, ONLY when following rigorous quality control standards during acquisition and processing in operation. LiDAR acquisition systems capable of recording lidar data with sufficient accuracy over a range of altitudes should be required. With good acquisition plan, highly qualified field personnel consisting of professional licensed land surveyors, licensed pilots and LiDAR technicians operate the system to ensure quality results from each flight to meet project requirements. Generally, laser data processing means numerous  'rigorous' working steps. Typical steps in their different working order are as follows: •Working with trajectories; •Dividing data into smaller geographical regions (blocks); •Classifying points by echo; •Deducing line numbers to points; •Classifying ground points separately after each flightline; •Measuring match of overlapping strips; •Solving heading, roll and pitch for whole data set; •Verify corrections visually; •Cutting overlapping point strips; •Classifying ground points back to default; •Starting final classification to ground, vegetation, building etc. classes. In practice, the data should be divided into smaller blocks of around 5- 10 million points, due to the limits of current operating systems and computing. Never delete points, add points, or change the elevation of points in LAS data, when working in the LAS format. Specialists only attribute each point with various flags that reflect attributes or characteristics of that point. Usually, returns are flagged several ways: by return number, by layer, or by type classes. Return number is simply first, second, third, fourth, etc., depending on the number of returns recorded by the particular sensor (attachment - Major LiDAR Sensors). Layer relates to return number, but takes one step toward elevation classification. In the LAS format, class types can be classified properly, even user-defined (attachment – LAS 1.4). If the end-result of the project is to produce a bare earth terrain model, the following categories are recommended: Bare Ground (Terrain); Features above ground (including buildings, tree crowns, cars, poles, bridges…); Water; and Noise Noise Removal The very first step in post-processing is to identify and eliminate noise points, which are extremely high or low points outside the range of realistic elevations for the project area. Anomalously high points can be caused by atmospheric aerosols, birds, or low-flying aircraft; low points might be caused by laser multipath. While noise points would probably be removed later by automated filtering, it is usually advantageous to remove them even earlier in the processing workflow. Many software packages use the absolute minimum and maximum elevations in a dataset as the basis for assigning a scale for color-by-elevation symbology. Noise points will cause the elevation range in areas of real interest to be compressed within the color scale. In addition to a simple band filter with high and low limits to classify these points as noise, which is Class 7 - Low Point (noise) in the LAS format, remote sensing specialists should use advanced algorithms to minimize noise, including manual editing. Once labeled, noise points can then be ignored by software for display purposes and in analytical computations, including feature extraction. Manual Classification As well known, automated filtering, appropriately applied, can effectively classify about 90 % of the ground points in a LiDAR point cloud. The remaining around 10% of the points must be visually inspected and classified manually, which involves human interaction with the data, familiarity with subject landscape, and knowledge of fundamental mapping principles, conventions,  "best practice" and high-resolution optical imagery (airphotos, GeoEye, QuickBird, Pleiades) with advanced Lidar package like TerraScan, ENVI LiDAR, Leica CloudPro (Leica XPro, Leica Cyclone), or ERDAS LPS (point cloud). In those LAS point cloud processing packages, manual editing and classification techniques are introduced to help improve data accuracy and reliability, in particular, when LAS data are not classified on latest LAS standard. This type of technique called 'Classify on Point Cloud' has been developed and is applied in the context of accepted mapping conventions and practices. For example, the Point Cloud Classify lets us classify point clouds based on parameters that define objects (man-made structures) and vegetation, including DTM, City Model (buildings), and Canopy model (lower vegetation, medium vegetation, higher vegetation). Pls refer to the paper called 'CLASSIFICATION OF LIDAR POINT CLOUD AND GENERATION OF DTM FROM LIDAR HEIGHT AND INTENSITY DATA IN FORESTED AREA'. The parameters for vegetation include a height and greenness criteria. The greenness criterion is applicable only to Point Clouds that have RGB information (below). RGB-encoded Points Imagery, after high-quality-controlled Acquisition and Processing (Accurate and Reliable LAS Points, ready for Geospatial and G&G applications) If the final result of manual editing  and classification is a detailed bare-earth terrain model (DTM), that means that those data have been quality controlled for completeness and lack of artifacts. If the final products include 3D feature clouds, advanced photorealistic rendering and 3D modeling techniques should be  applied to those 3D point clouds to create realistic representations and 3D analysis. For city modeling, 3D features (buildings, tree crowns, powerlines, tanks, cars, poles, bridges) can be classified and modeled. Finally, LiDAR intensity data also can be used to extract features, similar to raster imagery (see the slides by Bill)... Photorealistic Rendering from Point Clouds for Visualization and 3D Modeling ++++++++++++ The LAS file format is a binary file format that maintains information specific to the LiDAR nature of the data while not being overly complex (http://www.asprs.org/Committee-General/LASer-LAS-File-Format-Exchange-Activities.html .  Keep in mind that LAS clouds can be also generated from stereo-pair optical imagery, in addition to LiDAR. ... View more

Traditionally, there are three types of methods to fill voids in DEM like SRTM and InSAR, which are available in ArcGIS. A method to fill voids uses a variety of interpolators; a method to determine the most appropriate void filling algorithms using a classification of the voids based on their size and a typology of their surrounding terrain; and the classification of the most appropriate algorithm for each of the voids in the SRTM data. Obviously, the choice of void filling algorithm is dependent on both the size and terrain type of the void. Generally, the best methods can be generalized as: Kriging or Inverse Distance Weighting interpolation for small and medium size voids in relatively flat low-lying areas; Spline interpolation for small and medium sized voids in high altitude and dissected terrain; Triangular Irregular Network or Inverse Distance Weighting interpolation for large voids in very flat areas, and an advanced Spline Method (Topo to Raster in ArcGIS) for large voids in other terrains. However, on our practice, two latest methods (Fill and Feather method, Delta Surface Fill), which are only available in some leading Remote Sensing packages (below), are mostly recommended to do DEM/ DTM filling task in operation. ... View more

Hi, dear all, Can you share info on what the spatial data modeling tool is in your work?  Visio, Rational Rose, or others? We are looking for a visual and customizable spatial data modeling tool for ESRI environment. The goal is to entend or replace the CASE Tools extension in Visio Professional, so that we can combine both spatial and some of G&G models to integrate validated geological analysis into some geospatial workflows 'completely' in ESRI. Regards ... View more