ArcGIS On-line Credit usage

[GordonSumerling]

All,.

I am looking at the credit usage in ArcGIS On-Line. I have roughly 2.5 gb of features services which are stored in a database. On average these services are consuming 17 credits a day. I currently cannot find out which features are consuming the credits as it aggregates all the usinge to one number. Is there a way I can find out

a) which are my largest consumers of database storage. i.e. largest feature classes

b) which feature services are consuming the most credits

Regards

Gordon

[JoshuaFord]

Kelly,

When I run AGOL Catalog it list 9 separate content types.  All are hosted services on AGOL.

Do I have this breakdown correct?

Feature Service Storage (2.4 cr / 10 mb) =

Map Service 

Map Package

Feature Service

Tile and Data Storage (1.2 cr / 1 GB) =

Tile Package

Web Map

Service Definition

Image

Code Attachment

Web App

Thanks,

[KellyGerrow]

Hi Joshua,

I have updated your list below:

Feature Service Storage (2.4 cr / 10 mb) =

Hosted Feature Service

Tile and Data Storage (1.2 cr / 1 GB) =

Map Service

Map Package

Tile Package

Web Map

Service Definition

Image

Code Attachment

Web App

Only Hosted Feature Services (Published to ArcGIS Online are charged feature storage credits. If you were to reference a feature service from ArcGIS Server, this wouldn't count towards your feature service storage.

Let me know if you have any more questions,

-Kelly

[JoshuaFord]

Kelly,

This is driving me nuts.  AGOL Catalog results show 295 mb of total storage, 64 mb of which are Hosted Feature Services.  We're burning almost 10 credits per day from 1.44 GB of Feature Service Storage according to the My Organization page.  Where's the discrepancy?

Thanks,

[KellyGerrow]

Hi Joshua,

The sizes reported in catalog only reports to the size of the data of hosted feature services when they were published. Were any of the services published prior to July 2014 that contain attachments? This can be a cause for the discrepancy.

There is currently an enhancement regarding showing the current size of feature services available at this link

http://support.esri.com/en/knowledgebase/techarticles/detail/43869

Please Contact Technical Support for help in determining where the discrepancy is.

Thanks,

Kelly

[KellyGerrow]

Update:

The size reported on the item details page is the current size of data storage for feature services (not the size at the time of publishing).

[deleted-user-FfGbgR253rTy]

So... to confirm. If I've extracted the sizes from the catalog using something like agolTools from the ESRI github, then I should be able to calculate my hourly credit usage on an item by item basis.

If I've got the data in excel, then the formula to calculate this hourly credit usage would look something like this;

=IF("Feature Type Column"="Feature Service", "SizeinMB"/10*2.4/720,"SizeinGB"*1.2/720)

720 = the hrs in the month

Credits are charged at 2.4 credits per 10 MB per month, calculated on an hourly basis. ArcGIS Online | Service Credits 

The problem with this is that the size reported in the catalog for items created before July 2014 is not the actual size... (as per above) 

So... If we delete all the items created before July 2014 and republish any that we need, we should be able to create a broken down list of items and hourly credit usage. This could be re-calculated every hour to get a total number of credits used by an item...

The reason I'm looking at this, is that we currently don't have any way for our IT dept. who is responsible for purchasing credits, to recoup storage costs from the various groups using our organizational account. I'm sure we're not the only ones who need to be able to do break down these costs.

[deleted-user-FfGbgR253rTy]

So to update, I'm having some issues getting the above referenced formula to match with the reported credit usage in my organizational account status... So far, the closest I can get is if I double the number that the above referenced formula gives me. 

I have a call in with Tech Support to investigate, and will update this post when I get a response. I'm also hoping to expand the script to add other credit usage... 

[MikeMinami]

Here are some facts I think will be useful

• The size of a hosted feature layer as shown on the item details page now accurately reflects the actual database storage size for that layer.
• This search expression (I believe) will find all hosted feature layers in your organization if you are an admin and can see all private items.

  type:"feature service" typekeywords:hosted

Wallace, Cameron I'm not sure your expression is correct. We do calculate credit charges based on hourly database consumption. However, I think it would be difficult for your to do the same calculation. It would require that you have the total database size for each hour in the month to be able to calculate the fractional credit consumption for that hour, then sum all the hourly credit usage. Basically, the hour unit is the finest level of granularity we use to calculate cost, because database size is dynamic.The best you can easily do is determine the current credit consumption, based on the current database size.

We are working to improve our reporting abilities for item sizes so that people will be able to more easily find items that are consuming a lot of credits.This will eventually come.  In the meantime, you can get an estimate of the current consumption by going to your My Organization page, signed in as an administrator

1. As an administrator, navigate to My Organization.
2. Click on View Status.
3. Scroll to the bottom of the page and you'll see a circular credit utilization chart. Click on the segment of the chart that reflects storage. The chart will update to show storage.
4. Click again on the chart on the segment that reflects "Feature storage". Below the chart, you'll see a table that reflects the total database storage and file storage associated with feature layers and the credit consumption during the specified time period, which you set at the top of the page.

Thanks,

Mike

[deleted-user-FfGbgR253rTy]

Without the ability to recreate those credit charges on an item/individual basis, we can't continue to use ArcGIS Online (tragedy of the commons), as we need to be able to assign charges on an individual or project basis.

So, we're actually considering re-running the script every hour to pull the data from the REST API to store the granularity that we can't get from ArcGIS Online on our system.

The aggregated view does not let us effectively pass the cost of usage on to the project or user, meaning that it becomes an overhead cost which can and will be abused by the various users/projects, and is therefore not sustainable for us to use as an organization.

So... assuming the size of the reported feature is accurate, and that we're pulling the data from the REST API on an hourly basis, we, at least theoretically, should be able to recreate the credit usage total, but also support an itemized list of storage credits.

If the feature service is not hosted, it doesn't have an associated size, and is therefore not counted... so calculating credit usage based on whether or not the item is a "Feature Service" should still produce accurate results.

Currently, if I drill down as far as I can into the credit usage I can see that we're using 0.43 credits an hour, primarily in Feature Storage.

If I pull the data from the REST API using the above script, and calculate the credits used, I get, 0.404856765... So I'm close... but not quite correct. Once I can confirm that I have the correct calculation method, then it's just a matter of setting up a machine to pull the data and store/aggregate it for a monthly billing cycle. We'll also have to include credits used in other ways (i.e. enrichment, etc.). I agree this is difficult and painful, but our other option is to stop using ArcGIS Online, which I'm trying to avoid.