If you are interested in what happens on federal public lands, one of the most important databases available is the Bureau of Land Management's (BLM) LR2000 system, a non-spatial database with every record of oil, gas, renewable energy leasing, rights-of-ways, coal and other mineral development, land and mineral title, mining claims, withdrawals, classifications, and more on federal lands or on federal mineral estate. Every record has a legal description using the Public Land Survey System (PLSS), however who in their right mind has any understanding of how to read the PLSS!
Using Selenium as a web scraper and Python for parsing and batching, a team at The Wilderness Society has been able to shed light on the LR2000 system. They have developed a system to download and store the geographic locations and record information. They have developed oil and gas, coal, solar, wind, geothermal mapping applications that allow users to interact with the records from the LR2000 system that can be updated with a single click (almost).
This presentation will go over some of the technical details and challenges of using Selenium and Python to generate the spatial database as well as showing some of the big data results in a web map.