topic Re: Multiprocessing ERROR 000210 Cannot create output 'name' Failed to execute Intersect in Python Questions

Multiprocessing ERROR 000210 Cannot create output 'name' Failed to execute Intersect

JustinBridwell2 — Fri, 10 Dec 2021 22:06:51 GMT

I am trying to use multiprocessing to process intersects between polygons and points for a list of states and territories (the data sets for the points are large, so my hope was that the multiprocessing would speed the process up). But there seems to be an issue. Suppose I was just doing DC and PR. The intersect first produce an empty table output for PR and then throws the following error for PR:

Traceback(most recent call last):
  File"C:\Projects\hh_intersects\just_hh_intersects.py", line 68,in<module>    
  pool.map(multi_proc, state_list)
  ....
arcgisscripting.ExecuteError: ERROR 000210:Cannot create output  
T:\...\civis_hh_0520.gdb\PR_hh_intersects
Failed to execute (Intersect).

I'm not sure what I am doing incorrectly here. Any ideas on how to fix this?

import arcpy
import os
import time
import multiprocessing as mp
from multiprocessing import Pool, cpu_count 
# Paths for hh_pts_gdb, input_polys_dir, and output_gdb s
tate_list = ['DC', 'PR']
start = time.time()
p_count = cpu_count()
# Functions
def multi_proc(state):
    state_start = time.time()
    arcpy.env.workspace = output_gdb
    # Make output based on state
    output = os.path.join(output_gdb, "{}_intersect".format(state))
    # Pts
    pt_name = "{}_hh_pts".format(state)
    hh_pts = os.path.join(hh_pts_gdb, pt_name)
    # Polys
    st_poly = os.path.join(input_polys_dir, "{}/{}_UnionOutput.gdb/{}_Union".format(state, state, state))
    # Intersect selected pts with polys
    arcpy.Intersect_analysis([hh_pts, st_poly], output, "ALL")
    print("Output row count for {}".format(state))
    counter = arcpy.GetCount_management(output)
    print(counter)
    state_end = time.time()
    print("{} Process duration: ".format(state))
    print(state_end - state_start)

if __name__ == '__main__':
    pool = mp.Pool(p_count)
    pool.map(multi_proc, state_list)
    end = time.time()
    print("Total Process duration: ")
    print(end - start)


*Note- I have run the following by itself (outside of the multiprocessing) and it 
worked with state = "DC": arcpy.Intersect_analysis([hh_pts, st_poly], output, "ALL").

Re: Multiprocessing ERROR 000210 Cannot create output 'name' Failed to execute Intersect

JoshuaBixby — Thu, 21 May 2020 15:50:37 GMT

Looking at 000210: Cannot create output .—Help | Documentation :

000210: Cannot create output <value>.
Description
The output cannot be created. Potential reasons include data locking, an incorrect path, and limited access rights.
Solution
Confirm that the data is not locked by another user or application and that you have full rights to the workspace being used. Check to make sure that the path to the data is correct (check for typos in the folder path). Try creating the output in a new location.

Given you are using multiprocessing, it is likely that one process has a write lock on the file geodatabase while the other process wants to create a new feature class.

Re: Multiprocessing ERROR 000210 Cannot create output 'name' Failed to execute Intersect

JustinBridwell2 — Tue, 02 Jun 2020 13:07:03 GMT

Yes, it turns out that it was related to a lock. The issue was caused by the fact I was trying to utilize Mutliprocessing and simultaneously output feature classes to the same .gdb. Apparently, when you try to write the first feature class to that .gdb , it gets a schema lock the prevents the other 'workers' in the pool from doing the same. My solution was to simply have a new `.gdb` created for each output by state. Its a pain in the but to access them later but it allowed the multiprocessing to work and improved the speed of the operation.