"Get data from ArcPad" creates multiple identical records

[MatthewGibbons]

The situation:
I am using a file geodatabase to manage all weed mapping data at my office. It is being used by multiple staff members to collect weed data -  and was exported as an axf file ( so multiple users, multiple GPSs). 

The problem:
After a day of data collection ??? I upload the data using the ???get data from ArcPad??? function ??? which works great ??? uploads all the data on the GPS to the GDB. BUT, when you collect more data on that GPS the next day ??? and try to upload again ??? it uploads all the records from the previous day again, creating multiple identical records.  There does not seem to be away around this besides deleting all the records from that axf file on the GPS after each day ??? which is far from ideal. There does not seem to be a way of just uploading a subset of the data ??? ie - just that day???s records.

How it worked before:
In 9.3, I exported the GDB to the GPS units as a shapefile to collect the data.  Then, I used the Data Loader function in ArcCatalog to upload the data to the GDB ??? which allows you to setup a query to just upload a specific subset of the data.  This worked great.  Unfortunately, now, when I export the GDB in v10 as a shapefile for the GPS ??? none of the domain dropdowns appear, making it useless for GPS data collection. So, I switched to the AXF file format ??? which allowed the domain dropdowns to appear.  Unfortunately, the ArcCatalog data loader does not recognize axf files ??? so I cannot use this function any longer.  So, now I use the ArcPad Data Manager extension ??? ???get data from ArcPad??? button ??? which does not allow you to query the data  - and just uploads everything in that axf file ??? thereby creating identical records in the GDB.

Any thoughts?  It seems like there should be an easy solution here. Either figure out how to have functional shapefile data collection so I can continue to use the data loader, or figure out how to eliminate identical record uploads from the axf....

[DeniseKing]

Hi Matthew,

Typically duplicate records created in this manner are result of multiple 'Get Data From ArcPad' without the preceeding 'Get Data For ArcPad' function. Do you perform the 'Get Data For ArcPad' process prior to every 'Get Data From ArcPad' process?

http://help.arcgis.com/en/arcpad/10.0/help/index.html#/Overview_of_the_ArcPad_Data_Manager_for_ArcGI...

Cheers,
Denise

[MatthewGibbons]

Yes,
I did think of this as an option - though also not ideal.  It is somewhat time consuming to upload a refreshed copy of the GDB to each GPS each day.  Another option I can use to handle this is just uploading data at the end of the week - and then swiping the GPS clean.  This is also not a preferred way - but doable. 

Since there seems to be no way to specify what records to upload from AXF files - is there any way I can still use a shapefile version and upload with ArcCatalog dataloader.  Why has v10 disabled the dropdown domains in a shapefile that is exported from "get data for ArcPad"?  Is there any way around this? Being able to upload a specific subset of the data is a very useful function.
thanks.

[DeniseKing]

Hi Matthew,

Performing the checkout (Get Data For ArcPad) prior to each checkin (Get Data From ArcPad) is the required workflow or you will duplicate features if only using a single check-out and multiple chec-kin processes.

You are correct, there is not a method to specify what records to upload from  AXF files. However you can vote for this and other ArcPad feature enhancements via ArcGIS Ideas site, "Step through" ArcPad edits during check-in into ArcMap, http://ideas.arcgis.com/ideaView?id=087300000008OG0AAM.

We need to remember that only geodatabase feature layers truly participate in check-out/check-in processes. Shapefiles are strictly a copy-out process and is basically a copy/paste of shapefile, shapefiles do not provide transactional information which so many ArcPad users need, particularly in the QA/QC process.

You might look at using QuickProject Templates as an alternative workflow; I recommend reviewing the Help documentation and ArcPad Team Blog topic listed below before deciding upon this method. If your QuickProject is schema only, then duplicate records would not be created after using the QuickProject to Geodatabase tool to bring the ArcPad edits into the geodatabase.

Getting Started with a New QuickProject
http://help.arcgis.com/en/arcpad/10.0/help/index.html#/Getting_started_with_a_new_QuickProject/00s10...

Create ArcPad Templates
http://arcpadteam.blogspot.com/2010/09/create-arcpad-templates.html

QuickProject to Geodatabase
http://help.arcgis.com/en/arcpad/10.0/help/index.html#/QuickProject_to_Geodatabase/00s1000000wv00000...

The last item in your post asked "Why has v10 disabled the dropdown  domains in a shapefile that is exported from "get data for ArcPad"?" I'm not sure I fully understand the question, ArcPad 10 hasn't disabled dropdown domains in shapefiles when exported using 'Get Data For ArcPad' tool. Are you selecting an existing Layer Definition from a shapefile to use as a template for the check-out? The Layer Definition Templateallows you to reuse forms, scripts, icons, and other  layer definition elements (screenshot).

[ATTACH=CONFIG]6251[/ATTACH]

Get Data For ArcPad
http://help.arcgis.com/en/arcpad/10.0/help/index.html#//00s100000068000000.htm

If you need assistance with any of the above with your own project and are ArcPad maintenance is current, please log a support request with Esri Support and you can be connected with an ArcPad specialist.

Cheers,
Denise

[RolfBroch]

Hi Denise, do I understand you correctly that if you check out once and check back in more than once, you end up with duplicates?

From the help text:

"Once data is checked out the resultant ArcPad AXF file can be copied and distributed to multiple ArcPad users. Each ArcPad AXF file copy can be checked in individually, or as a batch, allowing multiple edits to be performed in the field by multiple users."

I thought that with the axf format you could check out once and check in many times. That is, you could either
- check in, keep on working and check in again...
or
- check in from several users based on the same checkout

Could you please elaborate a little more about the required workflow then?

Rolf

[DeniseKing]

Hi Rolf,

Yes, each check in (from ArcPad to ArcGIS) needs to be preceded by a check out (from ArcGIS to ArcPad) otherwise duplicate features will be created. I think the typical workflow diagram is an ideal visual representation of the complete processes.

[ATTACH=CONFIG]6301[/ATTACH]

http://help.arcgis.com/en/arcpad/10.0/help/index.html#/Incorporating_photos_into_your_workflow/00s10...

Cheers,
Denise

[RolfBroch]

Ok, but then I am confused, since Jian and Elvin had a presentation for ArcPad 8 on the UC in 2009 ("Introduction to ArcPad 8") where one of the benefits of geodatabase and ArcPad 8 was that you can check in and keep on working (see attached screenshot). This is also in line with the help text quoted in my previous posting. There is not much sense in checking in and keeping on working unless you can check in again.

Rolf

[DeniseKing]

Hi Rolf,

The workflow you describe is possible dependent upon users who continue to use the same AXF if it is the one that they checked in.  If they copy an edited AXF off their mobile device and check the data back in, they need to be sure to take the checked-in AXF and overwrite the one that is still on the mobile device.  If they do not do this, they will get the duplicated features because they will be working on a non checked in version of the AXF.  That is why we recommend the workflow I posted previously, as it is more straightforward and is less vulnerable to duplicate records being created. I hope this helps clarify my previous statement.


Hi Matthew,

I took your question, "Is  there any way to upload a specific subset of the  data from AXF during check-in?" to ArcPad team and they suggest that you query AXF file in Studio before check-in data process. In the example shown below, you should examine both AXF_TIMESTAMP and AXF_STATUS fields before and after check-in. It is the AXF_STATUS field that will be changed after check-in.

[ATTACH=CONFIG]6330[/ATTACH]
 
One (1) new pole was added to Riverside_mdb.axf. A SQL statement was used to query the data to see what had been added/changed today. I hope this additional information is helpful.

Thank you,
Denise

[MatthewGibbons]

Denise,
Thanks for the quick response. Although it seem like your suggestion using studio would work - I'm not sure if it's the right solution for my workflow (multiple users with varying degrees of comfort with ArcGIS).  I think we will have to use the check-in, check-out method. Nevertheless....

Regarding my prior statement about shapefiles and domains:  when I check out a GDB using "export as background data to shapefile (make editable)" option in the "get data for arcpad" function - the button for "select existing forms/scripts" is unselectable.  So, when I export this and put on GPS - the shapefile is editable in that I can take points - but all the domain dropdowns are completely blank - therefore I cannot take any other information associated with that point.

When I did this in v9.3 - all the domains from the GDB were automatically exported as dropdowns in the shapefile fields - I did not have to select anything extra - and I could collect all relevant data.  To me, it seems like ESRI is disabling some functionality of the shapefile to push us toward using the axf? or am I missing some crucial step here?

Thanks,
Matt

[JasonTipton]

Since there seems to be no way to specify what records to upload from AXF files - is there any way I can still use a shapefile version and upload with ArcCatalog dataloader.

You can export an .axf as a shapefile in ArcPad studio if that would help.  As for the intended check-in, check-out process, it is definitely a hassle. If executed perfectly, it works, but there are so many ways that it can go wrong.

If using a sync partnership:
When you sync your device, if ArcMap is open you may have problems. If you don't unplug the device after it syncs, you may have problems. If you forget to plug the device back in after check-in because you had to unplug it because if you didn't you would have problems... you will have problems. If you leave ArcMap open when you remember to plug back in your device to re-sync it, you might have problems.

The sync is just a huge hassle and data-loss just waiting to happen. I would suggest manually copying and pasting the old/new .axf files every time. Otherwise, you end up overwriting new data with old data.

If ArcMap could interact with the data straight from the device (like TerraSync and Pathfinder do) without having to rely on Microsoft Sync, the world would be a lot less complicated.