Classify Objects Using Deep Learning with VisionLanguageClassification

[cepsgis]

Howdy, 

I have been running into a bunch of issues trying to get this model to work. I don't really know what I am doing wrong I think it may have something to do with my ai connection file. I have tried with both OpenAI which generates output but it looks like it is not actually using my api key since it isn't consuming credits. After fiddling with it I switched to llama which just crashes ArcGIS Pro. 

Here's how OpenAI .ais file looks:

{
"service_provider" : "OpenAI",
"api_key" : "API_KEY"
}

Here's how the llama .ais file looks:

{

    "service_provider" : "local-llama"

}

@RohitSingh2 

[DanPatterson]

You have this question posted in the ArcGIS Living Atlas of the World place in Community.  

Is this your intended location? or are you looking to move it elsewhere like

ArcGIS Image Analyst - Esri Community

[cepsgis]

@DanPatterson  I was a little bit iffy on where exactly to post it. Maybe I'll cross post it. 

[SupratimBanik]

Hi @cepsgis,
To use OpenAI models, your .ais file should be structured as follows:
{
"service_provider": "OpenAI",
"api_key": "your_api_key",
"deployment_name": "gpt-4o" // Change this to the model you want to use (e.g., gpt-4o, gpt-4)
}

For using a local LLaMA model, your .ais file should look like this:

{
"service_provider": "local-llama"
}

To use a LLaMA model locally, follow these steps:

• Create a Hugging Face account at https://huggingface.co/join
• Open a Python command prompt and run huggingface-cli login. This will prompt you for an access token, which you can get from https://huggingface.co/settings/tokens.
• Visit https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct and accept the terms to gain access to the model.
• Run huggingface-cli download meta-llama/Llama-3.2-11B-Vision-Instruct in the Python command prompt to download the model.

Also after these steps make sure that you have got the llama weights at C:\Users\<username>\.cache\huggingface\hub\models--meta-llama--Llama-3.2-11B-Vision-Instruct location in your machine.

This should help you set up the models correctly. Let me know if you run into any issues!