I'm trying to upload a package to AWS Lambda and can not find/install the ArcGIS Python API for my package. Has anyone done this?
Thanks.
Figured this out...
Use Amazon EC2 and spin up a free tier Linux instance.
Install Python 3.6
Install Anaconda and use Conda to install the ArcGIS Modules
Zip up the install directory along with your Lambda_function.py file and upload it to Lambda (or s3)
Dependencies required for Lambda to work include ArcGIS, NumPy, Pandas and PyTz
Putty Tools were a good resource to work between Windows and Linux environments
Esri is releasing a docker container that has the ArcGIS API library and it's dependencies contiled in Linux. This container can be uploaded to an AWS ECR and run from there.
"Push to your private AWS ECR instance, and configure lambda to run from this container image. As of this writing, public AWS ECR instances are not supported for lambda."
https://github.com/Esri/arcgis-python-api/tree/master/docker
I was curious if anyone has recent success with this using the latest arcgis api module. The size of the site-package folder that gets filed up with ArcGIS API python dependencies seems to always be larger that was is allowed. Ive tried striping the package that would be uploaded to just the API module, but seems that lambda always complains of need additional modules in order to run
Seems like ArcPy is too big to fit into lambda restrictions:https://community.esri.com/t5/python-ideas/allow-arcpy-to-be-loaded-and-executed-through-aws/idi-p/1063441
There should be other alternatives to explore, such as AWS Fargate, or AWS Batch
@JonNordling1 ,
I believe if you use the checkOutextension function in the uploaded lambda script, it becomes a matter of access.
If the aws environment where the lambda is deployed has port access to the license manager, it should be able to grab the license for the script.
I have not verified this, but this are my thoughts on this.
Hi guys,
- Packaged ONLY in Linux env ? I was playing with Anaconda / Conda in window 10 ( all that virtual env. mimicking) and install ArcGIS package but no luck.
below is error from AWS Lambda with simple "from arcgis.gis import GIS" testing
- { "errorMessage": "Unable to import module 'main': No module named 'requests'", "errorType": "Runtime.ImportModuleError"}
Thanks
Is there an easier way of doing this without spinning up a Linux machine?
I am trying to run through the steps in this blog post, which suggests using PIP, but makes no mention of the machine needing to be linux.
However, we know that "AWS Lambda runs under a Linux environment"
Can we just download the linux package from Conda?
linux-64/arcgis-1.7.0-py37h39e3cac_863.tar.bz2 - is this the right one for Lambda?
And then upload a zipfile of this package into Lamdba?
Jon - I think you are looking for info on ArcPy. This thread is about Esri's other major Python API, called the "ArcGIS API for Python" (not a very creative name). ArcPy requires licensing, whereas the ArcGIS API for Python does not. That is more about working with server products, so you usually have to authenticate to the server, but you don't need any license to use the API itself.
One Question I would have is about the licensing, for example some of the scripts I would want to put in Lambda checkout extension. CheckOutExtension—Help | ArcGIS for Desktop
Does anyone know how this might look for Lambda since it would not have the ArcGIS License Manager Integration? This would be huge I think if there was a way.
These instructions from Esri should guide you.
install and set up | ArcGIS for Developers
For Lambda - I believe the package needs to be compiled in Linux environment for it to work properly. You'll need a linux machine to do so...you can create a new one on ec2.
For Existing Python - Virtual environments seem to be the way to go. Consider creating a new virtual environment and installing arcgis api through conda.
Bill Major, Daniel Hunter -
My question is not just about Lambda, but also integrating this into a preexisting python environment.
Do you know where this is on the Windows Anaconda install? Anaconda is in C:\ProgramData\Anaconda3 by following the installer's default. There is a folder C:\ProgramData\Anaconda3\Lib\site-packages\arcgis with a bunch of .py scripts. Is that folder all I need to grab, or are there other entries written elsewhere (like a registry of modules)? Does the arcgis API imports other modules I need to get?
-- For Lambda, I assume I just drop the arcgis folder under my scripts and zip it up, does that sound right?
-- For the existing python, do I assume I just re-create whatever conda created. I see there is a tarball in the Esri Anaconda channel. Is that a better starting point? All I could find on GitHub (via Google) were training docs and samples.
thanks!
Nice work Daniel. I too recently went through this exercise. One thing you might consider for the future is to create a Python virtual environment, pip install numpy and pandas, and then copy the arcgis module into this folder. This way, the only thing in this folder than needs to be zipped up is /arcgis, /numpy, and /pandas; not the entire Anaconda install folder. This should significantly reduce your upload package.
Nice job!
サインインしたメンバーは投稿、更新のフォローなどができます。初めてですか？無料アカウントを登録してください。
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
