Latest Contributions by RichardFairhurst

I got sidetracked with other work and have not finished the interface.  Anyway, I will see if I can get it moved back to the front burner. ... View more

I did a quick experiment by changing the inputs to allow the use of GPTableView instead of DETable.  The Join fields are recognized in the set up. However, the AddField_management tool required to add the case field fails, because a Joined table is not allowed as an input to that tool.  So I would have to write a subroutine to extract all of the join properties, break the join, add the field and then recreate the join.  I suspect that if I overcame that problem then there would be problems with the update cursor, since I would have to keep track of the unqualified and fully qualified field name at different stages in the code.  I also have concerns with the cursor performance if a join is in place.  If you are joining two or more tables at once then I am not interested in helping you, since I avoid that configuration at all costs.  So, you will have to give me a much more detailed description of the real life scenario you need to solve, since I am not willing to work out every possible complication that could arise with every possible join set up. ... View more

This is a Network Analyst problem if you are using a road network.  That requires a separate extension license to use it.  This post relates to the problem and how best to set it up, since it matters what is defined as an incident and what is defined as an origin. ... View more

If you created new fields and calculated them over with the field calculator then I see no reason why they would not show up.  Each time the tool runs it should recognize any newly added fields and their values in the single table.  It works for me that way.  So I would need screen shots to understand what you are seeing if it the new fields do not show up, since I find it hard to believe that newly added fields with values will not be recognized by the tool. Joined tables are very hard to set up and work with in ArcObjects.  The qualified field names are very tricky to use even when you can get them to work in ArcObjects.  I don't know what complications in creates when using Python. I set the tool up to use a Table directly and not a Layer/Tableview.  The inputs would have to be converted to a Layer/Tableview to detect any kind of a Join.  But all the rest of the code may still fail to deal with the fully qualified table names correctly if I made that change.  You would need to provide a more detailed explanation of what fields you are joining for me to know whether or not I want to even try to do what you want. ... View more

Given that I don't work with bus transit routing I don't have a clear idea of how these logs are organized.  What fields do your records currently contain?  How does it record time of day and distance information for each stop?  Are you looking to store an average for the total day of trips, average for the full round trip to a single location, average one way trips, etc.  Are there any variations in route for a given bus that need to be accounted for or weekday verses weekend differences?  Anyway, the dictionary key could be a tuple if you need to consolidate and organize data to include more information than just the bus, such as bus and date (regardless of time) or bus and stop pairs.  It all depends on how much you want to summarize the data at each stage compared to its current organization. ... View more

I would need more information to say whether or not that design is the most efficient.  Partly it would depend on where these calculations will ultimately be stored and how many records need to be evaluated to do the calculations.  Certainly you can design a dictionary to be structured as you have described, but given the size of your table this may risk using up all of the available memory if the entire table is loaded at once.  But without knowing what the calculations involve or where they will be stored it is hard to say what the best approach would be if memory limitations become an issue. ... View more

Append Management has a bug that won't refresh until you do things like manually remove the entire layer from the map and add it back.  You may find that you have a huge number of copies of the record created if you do that.  It does not respond to any refresh code when used in a script.  I got no help from esri support on the subject and had to abandon a tool I was developing based on using the Append tool in an interactive process.  It only works if the script is standalone and never needs to refresh an active ArcMap session.  I would consider using an insert cursor instead, although I seem to recall that it also has some quirks when trying to do a refresh in an interactive ArcMap session. I think esri filed a bug report.  You may want to let support know that you also expected the Append tool to work in an interactive scripted workflow, and that the inability to refresh the tool's output makes it useless for that purpose.  If they don't intend to fix it they need to document this behavior to avoid wasting our time developing tools that logically should work, but really won't if the user can only tell that the tool has actually altered the table by doing an unscripted manual set of actions after our scripts complete. I just reviewed my e-mails to esri support on the subject and see that they did not actually classify this behavior as a bug, but instead that the behavior you and I expect is an "enhancement".  Here is what they said in April of 2014: "After testing this behavior, I have logged an enhancement on your behalf to document this. The reference number and subject line for this enhancement is as follows: [#NIM100778  Ability to refresh the attribute table automatically after running the append tool when the target attribute table is already open. ] This enhancement will now be reviewed by our development team, and they will take further action as necessary to address this issues as soon as possible. For updates on this enhancement, please see the My Support portal at http://support.esri.com or contact Esri Support Services" I believe my original incident report only focused on the in ability to refresh a Standalone Table.  It may not have covered feature classes and the behavior of the Append tool in connection with refreshing the map view, so the NIM probably needs to be expanded. ... View more

I am glad that was caught, even if it means your last run of the tool was wasted.  It would be a pity to have made all of the improvements we made to the speed of your code only to introduce false data into the final result and analysis.  Al least another run of the current tool will be able to be complete for Monday.  I also appreciate the recognition and am glad I could help. ... View more

Laura: I am concerned that you made a change in your last version of your code that will produce incorrect lengths due to the Intersect tool and Summary you are now doing.  Before you were summing the lengths of the actual geometry created by the Clip tool.  That would correctly update any segments split by your envelope boundaries.  Now you appear to be summing a static double field called length_m that must have been populated before the Intersect tool was run, since I do not see any code that updates that field.  Unfortunately no static numeric field of lengths will be accurate after the Intersect tool is run unless it is first updated with the Field Calculator (which would be slow so you don't want to do that). No static numeric fields containing the length of the segment that existed prior to the Intersect will be automatically updated after the Intersect is completed.  A static length field will only be correct after the intersect if the complete segment was not divided by the boundary of any envelope.  All segments divided at an envelope boundary will still report the full original length of the original segment even though only a portion of the segment is actually contained in the envelope that divided the segment.  If a segment weaved in and out of an envelope the many divided segments within the envelope would accumulate the original undivided length of the original segment many times in the summary value of lengths for that envelope. The only accurate length values that the Summary Statistics tool could use after running the Intersect tool without running the field calculator would have to come from a file geodatabase feature class using the LENGTH field created and maintained by the geodatabase.  That fields will report the updated length for any partial road segment(s) within the envelope created by the Intersect tool.  Alternatively, a cursor reading the Intersect output directly into a dictionary could use the SHAPE@LENGTH field to do the same summary using the code I already provided. Both of those updated length fields will be reported in the original units of measure for your feature class, so if your units are not actually in meters you would have to multiply the summary value by a conversion factor to get meters.  A cursor does not allow you to specific a unit conversion factor directly to the reported geometry length values like the Field Calculator does.  There is a metersperunit property you can access from the spatial reference of your features provided you are using a Projected Coordinate System (I assume you are).   That factor could be used at some point in your code to do the conversion to meters.  The best time to do that would be during the creation of the dictionary holding the lengths, ... View more

Laura: The code looks good to me.  The dictionary could sum the length results while it is being created without using Summary Statistics if you wanted, but unless that step takes a significant amount of time revising the code would not make much difference.  In any case, I am sure that the bulk of the time is still taken up by the Intersect tool, but that step is probably as fast as it is going to be at this stage. I would be interested to know the time taken by the dictionary portion of the code relative to the time taken by the Intersect operation to determine if there may be other bottlenecks that could be optimized.  Reducing the time spent in each iteration by 1/6th would still result is a savings of about 2 hours if you are correct about the overall time for the code. Anyway, 12 hours is probably still taking too long to run on a nightly or weekly basis unless you have no other batch jobs you need to run. But the code provides a very acceptable overnight batch process if it only to be run once on an infrequent or monthly basis.  I assume that is all that this code is intended for. ... View more

Laura: The Intersect tool Darren encouraged you to try is the same tool I wanted you to use in the original thread you had created for this code.  I agree that this approach is likely to save a huge amount of time over processing intersect on each geometry separately.  My experience shows that the Esri programmers have optimized their own geoprocessing tools that have a specific function much more than they have the python geometry methods which have to be adaptable enough to be applied to all sorts of application situations the programmers have no control over. ... View more

The tool is written in VB.Net and I have not published the code, just the add-in.  I will be working out a process to add Z values to vertices of pipelines soon.  A GIS tech is plotting the pipelines and adding a starting Z and ending Z from our improvement plans into two double fields.  She is not editing the actual Z values. I will write a tool to add those Z values to the start and end points and interpolate the Z values for any vertices between using a constant rate of ascent or descent based on the 2D lengths between vertices.  For pipelines this should be fine since they do not follow any surface model and are only meant to change vertical direction where elevations are given.  Generally the pipelines are being broken where significant changes of direction occur, but only if an elevation is indicated.  True curves are being used for pipes that turn corners, so for those I usually only have to apply the start and end Z values. Anyway, I was waiting for a significant number of pipes to be plotted before developing the tool, but I think that point has been reached.  I have to develop my tools in .Net, because we use true curves, which python geometry methods do not support and would corrupt.  My tools also have to support both versioned and non-versioned SDE editing and non-SDE editing. ... View more

If you are trying to add Z values to the vertices between the beginning and ending point, then I don't have any suggestions.  However, if you have already assigned Z values to your lines based on your surface and want to calculate M values that will provide the 3D length of your line and allow you to determine the 3D length at any vertice or interpolate the 3D length between any vertices of your line you can download the Add-In I wrote from this post. ... View more

I meant for you to use the Intersect_Analysis tool and intersect all envelopes against all roads before doing any dictionaries.  Try the Intersect_Analysis tool.  It will work, I am almost certain.  This one envelope at a time approach is a bad approach.  I virtually never do that.  I really think that Intersecting every envelope once against every road will be faster and produce the same results.  Until I have proof that won't work I am not interested in individually processing the envelopes and think that is a waste of time.  Cliplayer is the complete set of the intersected roads against the complete set of envelopes and my code needed no modification (at least not until I know what the full Intersect contains). The code is reading every road segment of the Intersect output with an envelope sequence ID and summing the road lengths, which is all your code does as far I can see.  In reality this code happens before any other dictionary is created, and when you get to the part of the script that needs that length you just read the envelopeDict[sequenceID][0] value. I am asking for you to at least test the one feature set you displayed in your picture to get a processing time and to verify that the length reported is equivalent to your currently working slow process.  If it is meets your requirements for speed and accuracy, then there may be no real need to do individualized processing.  If I go to the trouble to build code for more individualized processing I will require that you have done this for a comparison anyway.  Otherwise you will not have done a complete set of optimization tests in my view and will have left out a critical alternative that I believe will beat any individualized process (until proven otherwise based on a comparison of speed or erroneous reported results). ... View more