topic Re: Reduce Storage credit consumption in ArcGIS Online Questions

Reduce Storage credit consumption

KhoajaKhaled — Wed, 15 Mar 2023 15:39:42 GMT

Hi GIS community,

Is there a way to reduce ArcGIS online credit consumption. I cleaned the unnecessary files and layers, but still it consumed most of the credits. What is your experience and advice.

Appreciate it.

Thanks

Re: Reduce Storage credit consumption

Richard_Purkis — Wed, 15 Mar 2023 15:50:36 GMT

Hi @KhoajaKhaled

Thanks for getting in touch

It is difficult to provide advice that applies to your particular organisation without knowing how much content you are hosting on ArcGIS Online and what your workflows require.

From my experience in Technical support I can point you in the direction of the documentation to make sure you:

Identify what services are using the most credits. Are these required?
Analyse credit consumption. Are credits being used for services other than feature storage?

Following on from that it may be worth having a discussion with you Account Manager as they will hopefully know your workflows and be able to advise if there are any better solutions for more efficient credit consumption.

Hope this helps

Re: Reduce Storage credit consumption

james-noble — Wed, 15 Mar 2023 16:24:18 GMT

Hi @KhoajaKhaled, Esri also recently introduced functionality to generate credit reports to help you understand which items are consuming the most credits: Report fields—ArcGIS Online Help | Documentation

Re: Reduce Storage credit consumption

MichaelVolz — Wed, 15 Mar 2023 23:44:55 GMT

Where is most of your credit consumption originating from - storage vs analytics vs other?

If storage is a problem, do you also have an enterprise environment where you can serve the data from on-premise source thru an enterprise service and reference the data in AGOL from the service URL so you would no longer be using feature storage credits?

I would look into seeing if some of your AGOL content can be obtained from external services so you could then delete that content from AGOL and free up storage. An example of this would be if you have floodplain data and hosted it up into AGOL. AGOL and/or Living Atlas has references to a FEMA floodplain service which you could possibly use instead.

Re: Reduce Storage credit consumption

ChristopherCounsell — Wed, 15 Mar 2023 23:30:16 GMT

Feature storage consumes 200x more credits per mb than other items. If you have large, complex layers (e.g. property boundaries for a country) they will likely consume the most.

There are a lot of great resources. Here is one that walks you through all of it - including how to identify and approximate the credits for each item.

https://esriaustraliatechblog.wordpress.com/2021/06/22/which-arcgis-online-items-are-consuming-the-most-credits-through-storage/

Re: Reduce Storage credit consumption

kreansanm — Wed, 22 Mar 2023 18:54:11 GMT

I used drive time analysis tools in map viewer classic, for 5-30 minutes driving time with 5 minutes interval. And just a single analysis used 90 credit. Is it common for this to happen?

Before doing this, I've tried doing similar analysis but that didnt take much credit.

The different between the two is the point file input. For one, I create the point file in arcgis online (didnt take much credit), but when I upload a zip file containing point shp to do the analysis it takes 90 credit.

Re: Reduce Storage credit consumption

ChristopherCounsell — Thu, 23 Mar 2023 03:07:39 GMT

For generating travel areas (also known as drive-time areas), you are charged for each travel area generated, so if you were to generate three travel areas (2, 5, and 10 miles) for eight locations, you would be charged for 24 travel areas. For more information about credit consumption when using analysis tools, see Understand credits for spatial analysis.

https://doc.arcgis.com/en/arcgis-online/analyze/generate-travel-areas-mv.htm

I would assume the credit amount is different because:

The number of features in the point file input is different
The number of service areas (minute intervals) is different
The service areas have a secondary analysis, such as geo-enrichment

Re: Reduce Storage credit consumption

LindsayRaabe_FPCWA — Thu, 17 Aug 2023 00:42:33 GMT

We're stuggling to reduce our credit consumption too at present. Ours is all around feature service storage (no analysis) and it seems that a lot of it is from 3 or 4 layers that have a lot of offline replicas (most of our data is offline enabled). Is there any way of reducing the consumption from those services without breaking the offline connections in our field staff offline maps?

Re: Reduce Storage credit consumption

ChristopherCounsell — Thu, 17 Aug 2023 01:35:08 GMT

How many maps?

Replicas and backend tables can be stored against the Hosted View Layer.

You may be able to take an alternative approach of:

Create new Hosted Feature Layer Views
Create new maps with the new Views
Encourage users to sync data (check in replicas) over a period and migrate to new maps
Unshare original maps and feature service / view (removing sharing over delete means you still have an opportunity to sync if users didn't bother syncing in step 3.
You can now delete other views, or delete replicas from the original feature service.

Toggling sync would achieve this but if you shared the original feature service it will require sync to support the views, so you would need to use python to drop all the replicas while still allowing sync on the new view layers. You'd also have the benefit of using Views moving forward. I always encourage people to never share the original feature service.

You could also manually or programmatically delete replicas but I'm not sure how you'd identify which are still in use.

Re: Reduce Storage credit consumption

LindsayRaabe_FPCWA — Thu, 17 Aug 2023 02:35:36 GMT

A bit of context from our org summary:

Thankfully we already use Views for the majority of feature services in our maps. I've just gone through and done some Change Log Trimming to see if that makes any difference of significance on the 3 largest feature services. I'm hoping it does!

We're also assessing what data we have in AGOL which may not be getting utilised enough to warrant the storage and removing where we can in full.

Re: Reduce Storage credit consumption

ChristopherCounsell — Thu, 17 Aug 2023 02:53:02 GMT

When you click into that dashboard to identify the biggest items, it has a bug where it combines the feature storage and photos. This may mean your '3 biggest' may be many GB but it's actually photos, and a lower-ranked item with <1gb of storage may be incurring 200x more credits in size

https://support.esri.com/en-us/bug/status-dashboard-for-credits-shows-combined-size-in-ite-bug-000141394

The credit consumption in your screenshot is correct but if you want to truly identify which items are consuming the most credits based on their size, you need to export the csv and break down both feature storage and attachments separately

https://esriaustraliatechblog.wordpress.com/2021/06/22/which-arcgis-online-items-are-consuming-the-most-credits-through-storage/

I have found in my experience it to be driven by:

Large volume of complex geometry (polylines and polygons with lots of vertices). e.g. non-generalized state/country parcels. Rivers datasets for country.
Editor tracking history. This grows substantially over time with large volume of editing.
Replicas (but again, mostly tied to #1)

Re: Reduce Storage credit consumption

LindsayRaabe_FPCWA — Thu, 17 Aug 2023 03:01:44 GMT

Thanks for the additional info. I can confirm for us it's the feature storage (and thus geometry) and offline replicas. We do have attachments on some layers but it's minimal compared to what many would have happening. We have been downloading the csv to review and track changes to individual feature services after trying different things. Trimming the change log for just our top 3 layers has saved us nearly 7GB!

Feature Service	MB (14/08)	MB (15/08)	Change (15/08)	MB (16/08)	Change (16/08)	MB (17/08)	Change (17/08)
FS 1	3604.9	3555.5	-49.4	3555.5	-49.4	1218.0	-2386.9
FS 2	3318.8	3318.8	0.0	3318.8	0.0	692.3	-2626.5
FS 3	2028.8	2028.8	0.0	2028.8	0.0	146.1	-1882.7