Prepare basemaps for offline use

22418
73
01-05-2016 07:13 PM
MarikaVertzonis
Esri Regular Contributor
7 73 22.4K

Survey123 supports the use of offline basemaps. These basemaps must be of the Esri file type of tile package and web mercator projection. Tile packages can be created using ArcGIS Pro.

NOTE: Currently there are many manual steps required to use basemaps in your survey. It is expected that a more streamlined process will be added, but we wanted to make the functionality available as soon as possible. This topic will be updated as the user experience of including basemaps in your survey, evolves.

Create a map tile package

You can create a map tile package using ArcGIS Pro. Once created you can copy it directly to your device for use in Survey123 or share it in ArcGIS for others to use.

To create a map tile package:

  1. Launch ArcGIS Pro.
  2. Open an existing map or create a new map. Ensure your map has a suitable name and description. To edit these, right click the map in the table of contents, and choose properties.
  3. In Toolbox, choose the Create Map Tile Package tool and enter the input map name and the output file name. Input map name and output file name are the minimum requirements for this tool. You can optionally enter other properties.
  4. Run the Create Map Tile Package tool.

      A new .tpk will be created in your chosen output location. To use this file in Survey123 you can upload it to ArcGIS to share with others, or manually copy it to your device.

Copy your tile package directly to your device

If you do not have internet connectivity or are unable to connect to ArcGIS, you can copy your tile package directly to your device, or use it from an external storage card.

To copy your tile package directly to your device:

  1. Connect to your device. If you are using an iOS device, you will need to use iTunes.
  2. Copy you .tpk to your ~\ArcGIS\My Surveys\Maps folder. For example, on Windows this would be C:\Users\UserName\ArcGIS\My Surveys\Maps.
  3. If you already had Survey123 open before copying, restart Survey123.

To use your tile package from an external storage card:

  1. Copy your tile package to your storage card and place the card in your device.
  2. Launch Survey123 on your device and on the My Surveys menu, choose Settings.
  3. Nominate an additional Map Library Folder that represents your storage card. The default path of ~/ArcGIS/My Surveys/Maps will already be shown. You can add additional paths by
    • Typing the additional path directly into the edit box (use a semi colon to separate paths).
    • Use the folder browser to select an additional path. Any path selected will be appended to your Map Library Folder.

     When you next open a survey that has the map append property set to true, tile packages on your storage card will be appended to the list of available basemaps.

Currently no checking is performed to ensure that Map Library Folders are valid. the first path in the list will used when you choose to download tile packages from ArcGIS Online. The Map Library Folder path will not appear on iOS as external storage is not supported on this platform.

Share your tile package

You can share a map tile package in ArcGIS so that you can download it on different devices, or share with other people to use.

NOTE: The maximum file size that can be uploaded to ArcGIS Online is 1GB.

To share a tile package:

  1. Sign in to  http://www.arcgis.com or your Portal for ArcGIS.
  2. Browse to My Contents.
  3. Select Create a new item.
  4. Select From My Computer.
  5. Browse to your tile package and choose to Upload. Be sure not to check the create tile service check-box when uploading your tile package.

     After your tile package has been uploaded to ArcGIS online, be sure to copy the URL of this package. An example tile package URL is http://www.arcgis.com/home/item.html?id=221a6aab56804a988a94ec9f82a063e7.

Associate a shared tile package to a survey

In order to be able make shared tile packages available to users of a survey, the survey itself needs to know where to download the survey from. This information is stored in the surveys .info file. Currently there is no user interface for this and you must enter text directly into this file. If you already have uploaded your survey to ArcGIS, you will already have a .info file that you can edit. If you have not already uploaded your survey to ArcGIS, you can create a new .info file with the following information.

To associate a shared tile package to your survey:

  1. Browse to your survey folder on your computer.
  2. Open the file MySurvey.info in a text editor. If you have not already uploaded this survey to ArcGIS, this file will not exist. You can create a new file called MySurvey.info, where MySurvey is the name of your survey.
  3. Copy the follow code snippet into this file and save.
  4. Upload your survey to ArcGIS.

   {
     "displayInfo": {
       "map": {
         "mapTypes": {
            "append": false,
            "includeLibrary": false,
            "mapSources": [
               {
                 "url": "http://www.arcgis.com/home/item.html?id=9a7e015149854c35b5eb94494f4aeb89
",
                 "storeInMapLibrary": false
               },
               {
                 "url": "http://www.arcgis.com/home/item.html?id=4497b7bb42e543b691027840d1b9092a"
               }
           ]
         }
       }
     }
   }

The following lists the optional properties you can define when managing tile packages in your surveys:

  • append—When you set the append property to equal true, the user will see the list of default online basemaps with your tile package appended at the bottom. When set to false, users will only be able to see the tile packages defined in your MySurvey.info file. The default value is true.
  • includeLibrary—When you set the includeLibrary property to equal true, all tile packages stored on the device will appear in the list of basemaps of the current survey. When set to false, only the basemaps (online or tile packages) defined in the MySurvey.info file will be listed. The default value is true.
  • storeInMapLibrary - Only relevant to tile package map sources. Specifies if the tile package will be stored in the map library or is private to the survey. If includeLibrary is set to false, then tile packages are automatically stored privately and this property will be ignored. The default value is true.

Download a shared tile package in Survey123

You can download tile packages for later offline use. Tile packages need to be associated to a survey for you to be able to find them and download.

To download a shared tile package:

  1. Launch Survey123 and sign in with your ArcGIS named user account.
  2. Select the survey that you associated a tile package too.
  3. Select the menu and choose Download Maps.
  4. Choose the tile package from the list to download.

     Once successfully downloaded, you will see the tile package in the list of maps on the location page of your survey.

Use a tile package in Survey123

You can use either manually copied tile packages or downloaded shared tile packages in your survey.

To use a tile package:

  1. Launch Survey123 and sign in with your ArcGIS named user account.
  2. Select the survey that you associated a tile package to.
  3. Choose Collect.
  4. Select the location question and display the full screen map.
  5. Select the menu and choose the tile package.
73 Comments
FasilTiru
Esri Regular Contributor

The sample tile package URL(http://www.arcgis.com/home/item.html?id=221a6aab56804a988a94ec9f82a063e7 ) mentioned in this blog is actually referencing to a tile service, not a tile package. Can you also please explain the two URLs in the code snippet? One is a tile package and the other one is a tile service.

Thank you!

Fasil T.

Esri Support

MarikaVertzonis
Esri Regular Contributor

Yes you are correct the first url is a tile service and shouldn't be there - sorry that was a typo!

try this one instead

http://www.arcgis.com/home/item.html?id=9a7e015149854c35b5eb94494f4aeb89

I have edited the original post with the correction, and also removed the erroneous comma.

Hot tip for anyone writing json - use an online json validator to check your comma's and brackets!

zkovacs
Occasional Contributor III

Hi Marika,

Do we have to use a specific projection/co-ordnate system for the map document an/or for the data/layers contained within? Ie. AGOL Basemaps use WGS 1984 Web Mercator Auxiliary Sphere. Do we need to convert our data into this system before creating the tile package for offline use? Or can we use national systems like British National Grid?

I tried the below:

1. Raster tiles in BNG, MXD in BNG >>> Empty squares when tapping on map window

2. Raster tiles in BNG, MXD in WGS 84 Web Mercator Auxiliary Sphere >>> Empty squares when tapping on map window (I did not expect this to work due to on-the-fly projection anyway.)

3. Raster tiles and MXD in WGS 84 Web Mercator Auxiliary Sphere >>> "Offline map preview not available" message in map window. Tapping on it does not do anything... Package is displayed in Map library under Setttings in the Survey123 app. (When I have multiple tile packages copied to the device, this particular tile package does not show up under the Map Types menu which is weird.)

Am I missing something? Do I still need to edit the .info file of the survey to associate the tile package with it?

(I'm using the Android version of the app. Online base maps work fine in the app, but we need offline base maps due to intermittent connectivity.)

Thanks in advance.

Zoltan

MarikaVertzonis
Esri Regular Contributor

Hi Zoltan

Yes Web Mercator is a requirement of the Qt map control (which is what Survey123 uses) - you need to use Web Mercator in your tpk's.

Yes you do need to hand craft the .info file at the moment, over time that will get rolled into the app and Connect, but its still early days.

Regards, Marika

zkovacs
Occasional Contributor III

Hi Marika,

Thanks for clarifying that I need to play with the .info file, that helps a lot.

Can we use on-the-fly projection in the MXD on which the tile package is based? (So have BNG layers with Web Mercator data frame with on-the-fly projection.) We do it with Collector and it works fine, but not sure if the control in Survey123 is able to handle it.

Cheers,

Zoltan

zkovacs
Occasional Contributor III

Right. I have some updates as I made some progress on it and others may find it useful.

It seems that we only need to edit the .info file if we are using a shared tile package/service stored in AGOL and want to associate that to a survey as Marika detailed above.

On the other hand, if you need to deploy basemaps in tile packages for offline use, there is no need to edit the .info file. I created a tile package (layers in British National Grid, data frame in WGS 1984 Web Mercator Auxiliary Spehere with on-the-fly transformation set properly) and simply copied it to the \ArcGIS\My Surveys\Maps\ folder on my Android device. It works fine now, although it may be a good idea to set a home location and zoom level for your project to be able to find your tiles easier. You can do this under Settings> Map> Home Location in Survey123 Connect when you open your survey.

You can have as many tile packages as you want (depends on your free space on the device) and use any of them in your surveys when offline.

Marika, could you please confirm that my assumptions here are right as well as creating/using the offline tile packages? I could not find any useful information about the on-the-fly capability of Collector nor Survey123, but I suspect they use the same map control. (Checking the geolocation stored in the feature service seems to be OK, so it seems that my tile package is okay.) Thanks.

Zoltan

MarikaVertzonis
Esri Regular Contributor

Easier summarized in dot points:

- Survey123 uses the Qt map control, Collector uses the ArcGIS Runtime map control.

- Survey123 currently uses no ArcGIS Runtime components - this makes it significantly smaller, and allows it to run on a much wider range of devices (eg: Survey123 on Windows Phone)

- Survey123 is built with AppStudio, and will soon be released as a developer sample in AppStudio.

- AppStudio does include ArcGIS Runtime components, so with the developer sample you can choose to swap in the ArcGIS Runtime map component in place of the Qt one, and have the same .tpk experience you have in Collector.

Regarding deployment of .tpk's:

- deployment is a copy / paste step as you describe. All .tpk's in the map folder will be available for all surveys

- editing of the .info file is required if you want to control which .tpk's are available in each survey. eg: you may not want to see every .tpk available on the device in a survey (you may want to force the use of just one)

zkovacs
Occasional Contributor III

Great, many thanks!

Z

CraigCarsley
Occasional Contributor II

My installation didn't include a My Surveys | Maps folder.  I only see My Survey Designs, My Survey Attachments, and AppStudio

I tried creating a Maps folder, but I'm still unable to make anything happen. 

Fasil suggested sharing the .tpk online, so I'm going to try that too.

by Anonymous User
Not applicable

Thank you for the detailed explanation! I managed to add the base map to the choice menu  but the geometry wouldn't show. I uploaded a tile package (I tried roads, plots and assets for different areas) to my content and referenced it in the info-file. 

Any idea how to make the data visible?

zkovacs
Occasional Contributor III

Craig,

The Maps folder created on the mobile device, see above in my post:

"...and simply copied it to the \ArcGIS\My Surveys\Maps\ folder on my Android device."

If you are using iOS, you still should have a similar location. (I don't have any iOS device to send you the path, but will send you later this week when I have a chance to look at one.)

zkovacs
Occasional Contributor III

Nicole,

Not sure what you mean on "geometry wouldn't show", but your tiles should be visible. If you cannot see them, you may need to configure your survey as I wrote it above:

"...although it may be a good idea to set a home location and zoom level for your project to be able to find your tiles easier. You can do this under Settings> Map> Home Location in Survey123 Connect when you open your survey."

First time I deployed a tile a package on the device, I couldn't see the content of it as they were "lost" at the small default scale. If you set the survey's home location to the centre of your tiles and modify the scale, the content of the tile package should show up without problems.

I hope this helps.

MarikaVertzonis
Esri Regular Contributor

Craig

If you are looking at folders on your desktop - which your description of seeing the My Survey Designs folder implies - be sure that you have installed Survey123 as well as Survey123 Connect. You will only get the My Surveys\Maps folder then.

Regards, Marika

by Anonymous User
Not applicable

Zoltan,

Thank you for your comment, although I am afraid it doesn't help.

I can see a tiles grid on my device but there is no data in it. I have set the home location to the area where I am. And also when I put a different base map to the background and zoom to the area where the tiles should be and switch back to my tiles I only see the grid no data. Zooming in and out does not help. I also tried setting different numbers (1 and 10) of levels of detail when creating the package. But it does not seem to matter.

CraigCarsley
Occasional Contributor II

Thanks Marika.  I didn't know there was a separate installation of Survey123.  I just installed it, and the My Surveys folder was created, but it only has a Databases folder- no Maps folder.

Will it still work if I manually create the Maps folder, or do I need to add the TPK somewhere else?

Could you offer an explanation as to how copying the TPK to this directory on my PC coordinates with the TPK on my device?

zkovacs
Occasional Contributor III

Craig,

I had some time to look at an iPad, but I'm afraid it is not obvious to me where to add the .tpk tile package in iTunes.

If I add it to the Survey123 Documents list, I cannot see it in the app on the iPad no matter what I do, the Map Library stays empty. MaybeMarika Vertzonis​ could clarify that.

(If I do the same with Collector, it works OK as the app moves the .tpk file to the Basemaps subfolder when I open Collector. But it does not seem to be the case with Survey123 - latest update.)

Survey123_iTunes.png

CraigCarsley
Occasional Contributor II

Thanks Zoltan.  It sounds like you've effectively reproduced my problem. I added the TPK in the same fashion (and also confirmed that it works on Collector), but I can't find it in Survey123.

zkovacs
Occasional Contributor III

Nicole,

I'll try to do a screen cast in the coming days about the workflow I'm following, hopefully that helps.

You do use the 'ArcGIS Online/Bing Maps/Google Maps' Tiling Scheme to set the scale levels when creating your tile package, don't you? You also need to consider the number of scale levels when creating it, as you'll only see the tiles in that range. (For example if you select level 1 only for the tiles, they will only display at 1:591,657,550.5). The below blog post details the scale levels used in ArcGIS Online/Google/Bing basemaps:

How can you tell what map scales are shown for online maps? | ArcGIS Blog

MarikaVertzonis
Esri Regular Contributor

Hi Craig - Yes you can manually add the Maps folder (and TPK) to that My Surveys folder. The folder on the desktop has no relation to the devices. You would need to manually copy your TPK to the equivalent folders on your devices.

MarikaVertzonis
Esri Regular Contributor

Sorry the chain of responses is getting a little tangled. Referring specifically to working with iOS, the maps do need to be stored in the same folder structure - the catch is that iTunes doesn't expose the full folder structure to you. You have two choices:

1) Use iTunes

         - copy the whole ArcGIS folder out of iTunes onto your machine

         - insert the Maps folder and your TPK's

         - copy the wholde ArcGIS folder back into iTunes

2) Use iExplorer

         - this product allows you to browse to a deeper level than iTunes. Copy the Maps folder and TPK's to the My Surveys folder.

zkovacs
Occasional Contributor III

I can confirm that choice 1 works OK. Although there may be cases when you don't want to copy the whole ArcGIS folder to your machine (especially if it's large). On the other hand, I had no luck with option 2, as I can only browse the photo folder (DCIM) in Explorer. It's the same on a machine without iTunes installed, so I may be missing something here. However, there are a number of apps you can use to see the file system on iPad/iPhone, thus you can copy&paste the .tpk file directly to the Maps subfolder. (I used iFunbox below.)

iFunbox.png

Tested, the tile package shows up and works just fine in Survey123 on my iPad. 🙂

TerriSutcliffe1
New Contributor II

I'm setting up a survey in Survey123 for the first time and happy offline maps are now available.

Could I please get some extra help for the Download a shared tile package in Survey123 section.

At step "2. Select the survey and choose Download Maps", I'm having trouble locating this option. Which screen is this on? Would it be possible to show a screenshot?

MarikaVertzonis
Esri Regular Contributor

To download a shared tile package:

  1. Launch Survey123 and sign in with your ArcGIS named user account.
  2. Select the survey that you associated a tile package too.
  3. Select the menu and choose Download Maps.
  4. Choose the tile package from the list to download.

DownloadMaps.png

FinnianO_Connor
Occasional Contributor II

If you are still having issues Craig, one extra thing to check is the name of the data frame in the MXD you are using to create the map tile package. I was having the same issue where the tpk file would work in Collector but not Survey123. What I found was that changing the default data frame name (Layers) caused the tpk not to be visible in Survey123. Normally I use the coordinate system as the data frame name but in this case reverting back to the 'Layers' name fixed the issue for me. Random!

MaryWasson
New Contributor II

I have been trying to follow the directions step by step.  I tried it with unchecking the check box when uploading.  I had a blank map.  I left it checked when uploading and this is what it looks like.

When I zoom in my data disappears.  (sorry for the double screen) What am I doing wrong? 

Thanks, Mary

DougBrowning
MVP Esteemed Contributor

I thought I had asked this already but can not find it.  Will there be support for unzipped TPKs also?  We unzip our imagery to make our app faster but then Survey123 can not find it.

thanks

MarikaVertzonis
Esri Regular Contributor

In what product (and version) did you create the .tpk? I suspect it something to do with how you created it.

DougBrowning
MVP Esteemed Contributor

My guess is you only built it down a few levels.  What did you set the cache level to?

MarikaVertzonis
Esri Regular Contributor

could you please clarify your request - the .tpk itself is a zip file. when you say 'unzipped tpk' do you mean 'a folder of images' (and not a tpk at all).

DougBrowning
MVP Esteemed Contributor

No I mean you take a tpk rename it .zip then unzip it.  Then you can use the Layers raster or the lyr files in there to view it.  It is much faster unzipped esp when the file is 3-5GB.

TimothyKing
New Contributor II

I created a tpk of 1 ft. contours at a scale of 1:895 with 10 levels of detail.  I added it to My Surveys/Maps folder and can see it in my map library, but it does not show up as a choice under map types  in Survey123 on a Windows Surface Pro 2.  Not sure what I may be  doing wrong.

Thanks

zkovacs
Occasional Contributor III

Is it a specific survey under which you cannot select the tile package? Or is it the same for all?

I would also double check that the package is correct by opening it in ArcMap - I can see from the screenshot that it was created in ArcGIS Pro... (You can also test the package in Collector if you have it.)

Hope that helps.

MaryWasson
New Contributor II

I created the .tpk in ArcGIS Pro 1.1  I have never created a .tpk before trying this.  I don't understand about setting the different cache levels.  I thought when I was zoomed in to my area of interest that was how it would show up in AGOL.  I tried opening it in ArcMap but it is blank in there also. Is there any way to do a screen share?  I'm so frustrated.

TimothyKing
New Contributor II

Thanks Zoltan!  I was actually able to get one of the tile packages that I created using ArcGIS Desktop 10.2.2 to show up in Survey123.  None of the ones I created with ArcGIS Pro 1.1 worked. I believe I also had to change the TOC to Layers to get it to work.  I also put some gibberish in the description and it looks like that is what the name of the tpk shows up as in Survey 123.  I also set the levels to 20 but I was hoping that my initial extent would be the first one and then would zoom in from there but it seems that the opposite is true, where my initial view is at a very small scale and then to a larger scale.  Not sure why the tpk from ArcGIS Pro did not work though.  When i opened it in ArcMap it didnt look right.

Thanks again,

LTK

by Anonymous User
Not applicable

Did you already try setting all levels of detail when creating the package?

That's what helped me.

MaryWasson
New Contributor II

Nicole, I'm not sure if I tried that or not.  You set the levels of detail in ArcGIS Pro?  I had to take a break from it for a couple of days till I could come back to it with a fresh perspective.  I will certainly give that a try. 

by Anonymous User
Not applicable

Hi Mary, You are right, sometimes it helps to have a break and start with a fresh view...

This is how it looks like in ArcGIS 10.2.

I had to set the level of details to 20 to have my data visible in all scale levels. Level 1 is like the earth view, level 20 is the most detailed one. Unfortunately you can't say you only want levels 15 to 20 or something like that. That would be far more efficient for the survey background from my point of view.

map_tile_package.jpg

Good luck!

MaryWasson
New Contributor II

I'm sorry for all the questions but can I do all of this through ArcGIS or do I have to use Pro?  I was under the impression ArcGIS Pro was the only way to create my .tpk for it to work in Survey123. 

by Anonymous User
Not applicable

Hi Timothy,

you should set the start scale in the settings-> map of your survey in Connect. There you can also define a centered point for your map. (Sorry, that's in German...)

The upper view is the one you get in the survey, the lower one when you click on the map to set your point.

zoomlevel_map_settings.jpg

by Anonymous User
Not applicable

No, I did everything in ArcGIS 10.2 and it worked fine (after setting the 20 levels...)

DougBrowning
MVP Esteemed Contributor

No ArcMap does it all.  Here is a good tutorial  Create Cache and TPK without ArcGIS for Server - Workflows for Desktop users

Also do not worry about it building levels 1-15 they are super tiny compared to the higher levels.

Also going to Level 20 will only work for small areas or it will get really big.  We usually stop at 16 which is a decent size/quality pay off (its what google goes to).  Some Esri products will "overzoom" a L16 down to L20 for you while other products go blank after your levels end.  Ni idea why the difference.

A mention that 10.4 now added a AOI parameter to the Create Map Tile Package tool which can really help the size for non contiguous area.  There are several bugs in the other tools when using AOI so I hope this one actually works - have not tested it yet.

zkovacs
Occasional Contributor III

Timothy,

I'm glad to hear that you managed to get it working. It's very picky when it comes to descriptions, title and data frame name...

TimothyKing
New Contributor II

Thanks Nicole!  I have about 30 separate sites that I need crews to go to and I want to create tpks for each one and then have them choose the correct one to use.  I am thinking that I can set the scales in general, but I will then need to crew to use the correct tpk file.

by Anonymous User
Not applicable

hm, of course that's tricky...

2016-02-25 22:31 GMT+02:00 Timothy King <geonet@esri.com>:

GeoNet <https://community.esri.com/?et=blogs.comment.created>

Prepare basemaps for offline use

new comment by Timothy King

<https://community.esri.com/people/tking_gp?et=blogs.comment.created> - View

all comments on this blog post

<https://community.esri.com/groups/survey123/blog/2016/01/06/prepare-basemaps-for-offline-use?et=blogs.comment.created#comment-21493>

DougBrowning
MVP Esteemed Contributor

You could use 1 large one with a Area of Interest parameter also.

TimothyKing
New Contributor II

Thanks Doug!  I am new to TPKs and the info you have shared has been really helpful.  I am not sure I understand how I could have 30 AOIs.  Thanks

LTK

DougBrowning
MVP Esteemed Contributor

I mean you build 1 map with all the areas in it.  Then use the AOI parameter so that it only builds the cache in those areas drastically reducing the size.  Then you just have 1 tpk to worry about. Of course there are some bugs in it  see it all here  Create Cache and TPK without ArcGIS for Server - Workflows for Desktop users

If that is too large it is pretty easy to script the tpk building. 

MarikaVertzonis
Esri Regular Contributor

I'm sorry if the comments here appear a little disjointed with this position in the thread, not quite sure where to reply, this is getting a little long!

- creation of .tpks in either ArcGIS Desktop or ArcGIS Pro SHOULD both produce .tpks suitable for Survey123

- in both cases, yes its picky about title and description for the map (ie: you have to have one)

- i have seen the Pro steps work on another machine, but I actually cant do it on my own at the moment. Some of you have observed that the .tpks that come out of Pro are slightly different - the difference is that (mine at least) they are missing a spatial reference - When I Create Map Tile Package why do i get a spatial reference of none in my tpk?​ I'm still trying to figure this out.

Thank you for pushing this very new feature so quickly. As described here, it is early days and editing the .info file manually is not something we want to keep as a long term solution, but enabled us to get the functionality to you quickly. In parallel to this discussion I'm taking all the useful info and adding the help, also working on some short videos to include. The new doc site will be ready for you very soon.

by Anonymous User
Not applicable

Hi again!

I tried now to save the tpk directly to my device. When I look in the data manager it's in the right place but it won't show in the list of available maps.

I understood then when I copy the tpk to my device I don't have to do anything with my info-file, right?

Any idea why the menu for the base maps does not show my file?

Can I limit the choice to the offline maps? In the upper description it is only mentioned that I can limit it to the data referenced in my info-file as I understand it.

by Anonymous User
Not applicable

Problem solved: we used the wrong coordinate system. At least with a smaller extent it worked now. I would have thought it would not be visible in the extent only when using the wrong coordinate system...