Scrape GIS data to find sites that match certain criteria

[theoryshaw]

I'm an architect (the building kind) that has been practicing open source architecture for a number of years.

All our 'blueprints' (CAD/BIM files) are open source and hosted on gitlab, github, and gitea.

We have a number of projects we've done in the past, (residences, multiple family, etc.).

Since we spent so much time developing these projects, it would be awesome if we could redevelopment them again, on another similar site.

The problem is, however, to manually find a site with the same (or similar) criteria (size of lot, setbacks, topography changes, density, etc.) as the original site would be near impossible.

Would be awesome if we could search/scrape GIS data to narrow down a list of candidate sites that matched the characteristics/criteria of the original site.

Relatively doable?

Perhaps a esri plugin?

[SpencerMathews]

As a current student of GIS I think answering your geographic question is what GIS is designed for. The type of datasets I would look for are building footprint polygons, survey data or reliable street data so that I could define accurate setbacks from the sidewalks, elevation data, and landuse zoning bylaw data to narrow down the permissible construction zones based off the type of building you are trying to find suitable locations for. With all these datasets in place for say, a whole city, then the process would be to use attribute and spatial queries to refine your search based off the specific criteria you want to apply. This project isn't just relatively doable, I'd say its one of the most applicable and doable type of GIS projects I could ask for.

[VinceAngelo]

I'd have to lean toward not doable.

The problem here is that there's too much data to scrape. Every city/county has its own GIS implementation, with a different mechanism for obtaining lot data, and different metadata available. I doubt you could accomplish this manually, in even places with the most sophisticated GIS operations available, much less in an automated fashion for all places.

Yes, searching an existing GIS database designed to support these queries would be trivial, but compiling such a database is thousands of hours of work, possibly hundreds of thousands of hours.

I doubt there an Easy Button solution here. If there is, it's going to be in a real estate forum, not in a Python programming one.

- V

[theoryshaw]

Thanks guys.

Specifically, this is the gis, in our area: https://dcimapapps.countyofdane.com/dcmapviewer/

Would there be enough here, I wonder.  

What if you started with a very limited data set.  Just scraping lot size, land use, and land/tax value?

[HaydenWelch]

If you're able to mirror the features you want in your own map, you can just use spatial queries pulled from your historical projects to pull data for that area. If their server supports REST queries, you can usually submit an extent and have it send you back features that are within that extent assuming you know the features you want to query.

You'll need the server rest url (usually https://<website>/arcgis/rest/services for a service directory). Definitely check the TOS for the data first or contact the GIS department that runs the server and see if they have any services you could use already set up.

From what I can see on that server you shared they have Parcel and Contour data and some tools for data export to ArcPro so it's definitely set up to do what you're asking.

[VinceAngelo]

That's a small enough set that you probably need to try it, though it still wouldn't meet your initial criteria, which included setbacks and topography.

There's also an issue with web scraping in general, since it's usually considered a violation of TOS for most web sites.

- V