Latest Contributions by MarcoBoeringa

Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once that which was once ArcSDE is gone. "Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once that which was once considered ArcSDE is gone." or "Hopefully, there will be a ArcPy tool that will register views with geodatabase metadata once that which was once the ArcSDE command line toolset is gone." might be more appropriate?... or are the ArcSDE system tables due for retirement too? ... View more

As a small testimony to the success of the described operation of enabling Personal ArcSDE / Desktop Geodatabase / ArcSDE Personal Server, see the attached screenshot showing the properties of the Database Server connection in ArcCatalog 10.2: ... View more

More importantly though, I don't want a forum.  I want an Esri help page where I can get some basic information about what they are, how they are licensed, what they are used for (vs other GDBs), and how they are created or installed. ... The problem with the forums is all I'm seeing (when I use Search the forums on Personal ArcSDE with no quotes, but presumably NOT updating to All Collections, but still returning 1500 hits) is discussions of Personal Geodatabases where SDE is mentioned, and the few hits with Personal ArcSDE are unsolved problems (at least on the first page of 25 entries).  I want an Esri help page with the title: Personal ArcSDE. About Personal ArcSDE (also called "Desktop Geodatabase" or "ArcSDE Personal Server" in some documents by ESRI): I fully agree this is badly documented, but on the other hand, once you realize there is actually no real distinction between Personal, Workgroup, and Enterprise Geodatabases, except a licensing scheme determining the maximum no. of concurrent users for the geodatabase (Vince may protest here, but for the sake of this thread, I think it valid), it actually turns out to be surprisingly simple: *** STEPS TO INSTALL "PERSONAL ARCSDE / DESKTOP GEODATABASE" *** (E.g. with "ArcGIS for Home Use" license based on "ArcGIS for Desktop Advanced") * EDIT: * It may be advisable to reboot in between / after some of these steps, e.g. installing SQL Server, so as to ensure all services are properly running and usable for the next step. 1) - Install SQL Server 2012 Expresss I am not going to detail all steps of the SQL Server installation here, use the Microsoft documentation if in doubt. Basically, you can choose almost all defaults, as the 2012 version of SQL Server installation already sets it all up under restricted Windows accounts, so rather safe. The defaults will install "extra's" like the Reporting Services that aren't necessary for the functioning of an ESRI Geodatabase, so you may prefer to opt out on these "software developer / SDK options" of the installation. NOTE: you can also use the ArcGIS for Desktop 10.2 installation to install SQL Server 2008 R2 Express, as it is included in the desktop installer, but since I guess you will want to use the "latest", install 2012. 2) - Create a new instance in the process of the installation of SQL Server. Record the name of the instance for later use. 3) - Ensure you are logged in as a full Windows Administrator on your PC. This is very important, if you don't do this, the enabling of the database for geodatabase storage may fail. Make sure this administrative login is also registered with your SQL Server Express if you haven't already done so during the install of SQL Server Express (which is usually the case, so you can probably skip this). 4) - Start the ArcGIS for Desktop 10.2 installation. NOTE: Yes, you need the installation for ArcGIS for Desktop, not ArcGIS for Server. So you can use the installation that is part of the 10.2 Desktop Advanced "ArcGIS for Home Use" program. It is downloadable from the Customer Care portal once you receive your license from ESRI. 5) - Once in the main screen, choose ArcSDE for Microsoft SQL Server 2008 R2 Express as the installation option. NOTE: don't worry: it won't automatically overwrite or re-install your SQL Server 2012 Express, see next steps. 6) - Once the install wizard appears, choose Next 7) - In the next dialog that appears (Installation Options), deactivate the Microsoft SQL Server Express Edition option, since you already installed it in step 1). Leave the Enable geodatabase storage on SQL Server Express checked. You want your database to be enabled to use ESRI Geodatabases. 😎 - Click Next 9) - In the Choose a SQL Server Instance and add a Windows login dialog, choose under SQL Server instance name your instance created in step 1), e.g. YOUR_COMPUTER_NAME\YOUR_INSTANCE. Add a valid Windows Login who will administer the geodatabase, e.g. YOUR_COMPUTER_NAME\YOUR_LOGIN_NAME. Note the YOUR_COMPUTER_NAME, since we are using Personal ArcSDE on SQL Server Express, your local PC is likely the server. 10) - Click Next and finish the installation. You can also close the ArcGIS for Desktop installation window now, as you no longer need it. Your database should now be enabled for creating geodatabases 11) - To create a geodatabase, DON'T USE THE "Create / Enable Enterprise Geodatabase" TOOLS!!! I can not emphasize this enough. As the name of the tools already indicates, these tools only work against "enterprise" databases, not on SQL Server Express! 12) - To create an ESRI Geodatabase on SQL Server Express, simply create a new Database Server(!) connection to the SQL Server instance in ArcCatalog using the Add Database Server option under Database Servers in the Catalog Tree of ArcCatalog. NOTE: I wrote Database Server, not Database Connection. You must first create a Database Server connection, as only a Database Server connection contains the context menu option to create a New Geodatabase, as in step 13). 13) - Now right click the new Database Server connection, and choose New Geodatabase from the context menu. Yes, it is that simple! No hassle with an authorization file as in the "Create / Enable Enterprise Geodatabase" tools. The authorization of ArcSDE Personal Server forms an integral part of your ArcGIS for Desktop license, and was applied in steps 4) to 10). NOTE 1: In order for this step to work, you must have an ArcGIS for Desktop Standard or ArcGIS for Desktop Advanced license. This doesn't cost you anything extra with an ArcGIS for Home Use license, it is just another licensing scheme you need to activate / authorize from the ArcGIS Administrator included with ArcGIS for Desktop. In Desktop Basic the New Geodatabase option is disabled in the context menu. NOTE 2: After you have created the geodatabase via the Database Server connection, you can now also use the Add Database Connection option in ArcCatalog, to create a new Database Connection to your geodatabase. Congratulations, you have just created your Personal ArcSDE / Desktop Geodatabase in SQL Server 2012 Express! ... View more

So you're pretty much my new hero for thorough post replies now. If you were invloved in the above, kudos to you! That is a pretty amazing achievment. No, I wasn't involved in this particular project. I just thought it worth mentioning, as I just discovered these articles myself (I wasn't aware of this ongoing project), and thought it worth mentioning as being quite a remarkable achievement. I have in the past been involved in other major projects here in the Netherlands for the Ministry of Transport, including a country wide highly detailed (about 1:10.000 scale) traffic noise modelling program and database implemented using ArcGIS, geodatabase versioning and a custom 3D traffic noise modelling core build by our project partner. ... View more

unfortunitely the powers that be did not want the data manipulated from the what was delivered. i will have to bring this up with them again. Somewhere accuracy is going to be lost (from the original delivery) if they want it in SDE. may it be in the resolution of the coordinates or in the reduction of verticies. I can think of no reason not to generalize contours from a (LIDAR) DEM or point cloud. If you want the accuracy of the original data, start using the original data... I see little value in terms of analytics, or scientific analysis, in height contours putting such severe constraints on accuracy that generalization would be out of the order. Height contours are usually just cartographic (but maybe there's the culprit, and your customer is a cartographic agency afraid for quality loss compared to traditional "hand-drawn" photogrammetric contours...) If the latter is the case, it maybe of help to you to hear that here in the Netherlands, the major cartographic agency responsible for country wide topographic maps (the "Kadaster"), actually just managed a unique achievement, that even got them an ESRI "Special Achievement in GIS" award: implementation of a fully(!) automated cartographic generalization process based on a huge - 400+ models - custom build ArcGIS Modelbuilder / FME suite that creates 1:50.000 maps from 1:10.000 without human intervention. This seems a world's first and quite a colossal achievement, especially since the 1:50.000 auto-generated map is fully replacing the existing "manually generalize"  production line for 1:50.000. There is no other cartographic agency who seems to have achieved this yet, although many have research projects in this direction. A Dutch article about this: http://www.gdmc.nl/publications/2012/Automatische_generalisatie.pdf And from an online publisher an English language article in "Cartography and Geographic Information Science": http://www.tandfonline.com/doi/abs/10.1080/15230406.2013.824637#.UnK83-L27zs ... View more

I have a contour feature class stored in a file geodatabase. when looking at the contents tab of the geodatabase in ArcCatalog i see that the size of the file is 4GB. the contour data that we have was generated by our vendor from a LiDAR point cloud. I'm gonna check on the accuracy of their collection to better justify the decreased resolution of the coordinate system. That is a rather huge vector Feature Class by any standard. You really have to wonder, especially if this contour data was auto-generated from a highly detailed raster DEM or LIDAR point cloud, if there isn't a huge amount of bloat in this dataset in terms of excess polyline vertices. Usually, with auto-generated contours, a generalization step is in order and can often reduce storage constraints considerably, while maintaining the basic quality of the dataset. There is often way to much detail and vertices added to the contours in the contour generation process. Generalization will probably also mean you don't have to reduce storage precision, while still achieving a huge reduction in dataset size. hahahah. sorry. It would be Enterprise, licensed through ArcGIS for Server. I really think you should be looking at getting some sort of "enterprise" type license for your SQL Server as well. It is a waist of money to have an Enterprise license for ArcGIS for Server, while using SQL Server Express as the back-end. ... View more

From what I read here based on remarks by ESRI employees, the next version will be 10.2.1 and should come out somewhere at the end of this, or beginning of next year. But only ESRI can tell, and answer if it will support 8.1 as well. You may need to wait for 10.2.2... See this post in another thread for a remark by an ESRI employee. And this thread may also have some relevance for you, although it is about ArcGIS for Desktop, not ArcGIS for Server. Since it seems you already successfully installed some stuff, it is probably unrelated. Anyway: Problem with installing arcgis 10.2 to windows 8.1 9600 ... View more

Windows 8.1 is not yet officially supported (8 is for testing purposes, but not as real deployment platform): ArcGIS 10.2 for Server system requirements You will probably have to wait for the next release, or find another machine that does meet the requirements. ... View more

I have no clue how the data was migrated. This was all handled by our db admin team. The database is not anormally big and as for the Oracle version, apart from 11gr2 I couldn't tell you. I'll pass those questions to the database administrators as well as the tip for data fragmentation. If this all happened through DBA's not aware of GIS datasets and the potential spatial fragmentation, than there is a real possibility tables got mangled / sorted in an undesirable way. As Vince wrote in one of the other linked threads, there is an easy way to get some idea of a potential problem: "The hallmark of spatially fragmented data is when rendering (edit: viewing the data in ArcMap) produces a random-seeming draw order, vice filling from one side of the field of view (edit: dataframe in ArcMap) to another. There is no silver bullet to solve spatial fragmentation, but there are techniques available to moderate its influence." ... View more

1. I couldn�??t figure out the effect of �??update cache automatically�?�. Doesn�??t this mean that the cashed service is updated automatically once the source data is edited? Jamal, that is a good catch and one I think ESRI needs to answer or at least document better, because I am now confused what the Update cache automatically option is actually supposed to do: determine the way the one time creation of the cached map service is done, or truly update your cache automatically. Beats me based on the documentation... it seems the "Update cache automatically" may actually be a very bad description of what this option does. Please see the: Map service caching configuration heading on this WIKI GIS page maintained by ESRI. Scroll down on that page to see the heading I meant. It seems this option has little to do with "updating" your map cache automatically when something changes, but rather refers to a choice to use an available default caching service for creating the caches, or to define manually your own running caching services (server computers you dedicated for the heavy job of creating cache tiles). Please bear with me I am entirely going by the documentation here~! I don't have a running ArcGIS Server available, neither generated caches myself. It is going deep here into ArcGIS for Server, but you may wish to read this Help page too: Allocation of server resources to caching 2. �??The create cache on demand�?� sounds not to solve the issue of updating the tiles neither. The database is updated while the cached services are not Jamal, you can probably not properly test this feature with just a small dataset. If the dataset is so small so as to require only one tile, your tile will be generated on the first view. If you subsequently update some portion of your geodatabase in an area that is still within the bounds of the original dataset, as you do here, there may be no trigger to generate a new tile, as there is already a tile for that area. I don't know if the tile is supposed to update or not automatically based on an edit, that may require the other option "Update cache automatically". Also mind this option from this Help page: "Allow clients to cache tiles locally When this box is checked, Esri client applications, such as ArcMap, are allowed to keep a local cache of any tiles they retrieve from the server. This can help the client view the tiles more quickly when returning to the area. Clients must manually clear their cache to get any newer tiles from the server; therefore, consider unchecking this box if you'll be updating your cache frequently." I also still think you may be misinterpreting or mis-using the caching features. Caching is only really appropriate for complex maps, where the cache represents multiple, or complex layers, including imagery, highly detailed vector Feature Class data (e.g. scale 1:1000 - 1:25.000), or data represented using complex cartographic symbology or intelligent labelling. I do know you use aerial imagery too, for which using map caching - especially in combination with overlayed Feature Classes for for example roads or buildings - may be entirely valid. In all other cases (non-complex Feature Classes, basic labelling), you may be better off using a dynamic Map Service, of even a Feature Service. As a reminder, from this Help page: "Can I cache all my maps? A map cache represents a snapshot of your map at one point in time. Because of this, caches work best with maps that do not change frequently. These include street maps, imagery, and terrain maps. If your data tends to change, you might still be able to use the caching tools to periodically update the cache. You can even schedule these updates to occur automatically. To understand whether your frequently changing map can still be cached, it helps to ask these questions: How up-to-date does my map need to be? If the data you see on the map needs to be live, with no time delay acceptable, caching is not appropriate. However, if a short delay is acceptable and the cache updates can be performed within that time window, you can still use caching. How big is the cache and how widespread are the changes to my data? These two questions go together. A large cache takes more time to create. It may be practical to update your large cache only if you can isolate the changed areas and update those alone. If the cache is small, you might be able to quickly rebuild the entire cache. If the update cannot keep up with the changes in an acceptable amount of time, the map is not appropriate for caching. After considering the questions above, use caching whenever it's appropriate. The performance benefit that you gain is probably worth the investment it takes to create and update the cache." ... View more

The problem may actually be that, although you think the attributes of the original data and the new domain values are the same, the domain values most likely use numeric values or short text codes internally, and only show you the longer, human readable, label or description as a more user friendly substitute. This means you need to convert your original data to the appropriate numeric or short text code values as you defined them yourself during domain creation (under "Coded Values" / "Code" and "Description" in the dialog), to have them show up with the correct description in the tables. E.g.: If you had an original field named "City" with already entered values "New York" and "Chicago", and you created a domain like: "Code", "Description" 1, New York 2, Chicago and you subsequently associated this domain with the field "City", than the original text values "New York" and "Chicago" wouldn't display, since the allowed values for the field are 1 and 2, not "New York" and "Chicago", from the moment you associate this domain with this field. If you changed the original values to 1 and 2 respectively, the attribute table would show the domain descriptions "New York" and "Chicago". ... View more

Recently, our database servers were migrated from Oracle 10g to 11gR2. Immediately after this migration we experienced a rather spectacular rise in response time from the ArcGIS Server. The identify task seems to be the most affected functionality with response time up to 10 times longer than they used to be before migration. What method for migration of the spatial data did you use, and how big are your datasets? What exact versions of Oracle have you been using? If the data somehow got spatially fragmented during migration from 10 to 11 (features spatially nearby no longer physically close to each other in database tables), while the data was rather coherent in the original 10 database, that might be a cause of the worse performance. Spatial fragmentation is not an issue by itself for the functioning of the geodatabase in the sense of being able to effectively find and retrieve features (after all, that is what spatial indexes are also meant to deal with), but it may still make a significant difference in the amount of physical hard disk activity, swapping from record to record to retrieve data within a certain spatial context. See this thread also for more details: ArcSDE Performance Large Databases You may also find this thread useful: Slow Perfomance of ArcSDE ... View more

Hello, We are in the process of setting up several SQL Server Geodatabases; and, are faced with IT restrictions at the SQL Server security level.  Namely, no database/mixed-mode authentication.  All connections must come via Active Directory. The approach we would like to take, given these restrictions, is as follows: For the SDE geodatabase owner: 1.  At the instance level, grant an AD group access to the instance. 2.  At the db level, map that AD group to the sde user/sde schema in the db/gdb. I think you need to map that AD group to a SQL server database level role, not the SDE user / schema. You should never store your user data in the SDE schema. I think Vince Angelo gave enough hints in that other thread you linked to get you going, especially: "Keep in mind that they're tables, not files. Tables have to have an owner. The group grants access, but the user owns the tables. Best practice is to create one or more users to own the tables, then grant access to users (or groups) through roles." and: "Local security guidelines would of course take precedence, but the owner account would generally be a "headless user", and therefore better map to a DBMS login (named for the theme, department, or data source, e.g., "Basemap", Transportation", "USGS"). The password would be closely held, to make sure that it was only used by the data librarian for changes to content and database design that have been approved for use within the system." And as the Help page Adding Windows-authenticated logins or groups to a SQL Server database also states about the specific case of Windows groups: "For Windows groups, a matching schema will be created for any group member the first time the user creates data in the geodatabase. The schema name will be the same as the login name, not the name of the group. This is done automatically; you do not have to create the schema ahead of time." So for Windows groups, a user schema will be automatically created once a user that is part of the group, creates data in the geodatabase. ... View more

Turning on\off layers and updating of the cached services, Sounds that there are two issues with the cached (tiled) services 1. When caching an mxd file that contains multiple layers, then these layers can�??t be turned on\off on the web mapping application. Does this mean that we need to create a separate service for each single layer to be able to turn it on\off from the web mapping application? Jamal, Cached map services are just "dumb" images, like a JPEG photo from your iPhone, even if the original data was vector / Feature Class data. In this respect, they do not differ from "image services". They do, however, support things like identifying based on a course back to the original vector data in the geodatase. So yes, if you include multiple layers in a single cached map service, there is no way to turn on/off a single layer, as all layers are in that "single" image (albeit cut up in many tiles). As you deducted, you need to create a separate service for each layer if you need that turn on / off ability for each individual layer. 2. How the cached services are updated as the data changes? Do we need to �??overwrite an existing service�?� when the data changes (edited)? In case of dynamic service, we don�??t need to �??overwrite an existing service�?� as the data edited, the edits are transferred to the web application directly. As for the "dynamic service", this is because you are always directly reading the original data, not a copy that may become "outdated" if not explicitly refreshed / renewed. You may consider using the Create tiles on demand option, as described on this Help page, and in detail on this Help page: Map caching on demand. This overcomes the need to pre-generate your entire caches, since it creates tiles on the fly based on user demand, to be stored for later usage by other users. This allows you to gradually build up the cache instead of an entire cache in one go. Also mind this warning in the Help about cached image services, that probably also applies to cached map services based on Feature Class data: "If the update (edit: of the caches) cannot keep up with the changes in an acceptable amount of time or the image service has to be changed frequently with different image properties, then the image service is not appropriate for caching." 2. How the cached services are updated as the data changes? Do we need to �??overwrite an existing service�?� when the data changes (edited)? See the Updating tiles topic on the Help page I linked before: Map caching on demand ... View more

John, I can't help you with the exact cause of your observation, but in general about Kriging and interpolations: - Since kriging relies on modelling the semivariance, it requires a minimum set of datapoints to get any reliable result. You only have 9 sample points, which is a very low number. I really think your dataset is to small. This is also confirmed by the document I attached. It is a nice introductory document to Kriging and Geostatistics written by Gregg Babish of Environment Canada ("Geostatistics Without Tears"). It states in one of its paragraphs: "Variograms computed from small samples almost always appear erratic, and in many instances the principal cause is too few data. It has been shown (Webster and Oliver, 1992) that variograms computed on fewer than 50 sample points are of little value and that at least 100 data are needed." - I would strongly urge you to refrain from using the "Simple Kriging" options of Spatial Analyst and 3D Analyst for any real "scientific" application, as it has only very basic options, and doesn't give much insight in what you are doing. I would urge you to use Geostatistical Analyst instead, which offers far more and reliable (visual, graphical and statistical) feedback on your chosen settings and interpolation parameters. The Geostatistical Analyst Help and some documents here on the ESRI website, also give good introduction to kriging and geostatistics. *** EDIT *** I now noticed there is slightly more recent, and extended, 2006 version of the same paper by Gregg. You can download it from iBrarian: http://www.ibrarian.net/navon/page.jsp?paperid=15417085 ... View more