Troubles with Learn Lesson and bridging data into R

[KaylaFlamm]

I am attempting to complete the Learn Lesson Perform ecological niche factor analysis in R—Identify an Ecological Niche for African Buffalo but am having troubles with bridging my data into R. Specifically, step 6 onward: the following function exactly as it appears in step 6 of the lesson simply does not work for me: arc.raster(data_path). When I attempt to enter this in RStudio, I keep getting the message Error in arc.raster(data_path) : object 'data_path' not found

I am running ArcGIS Pro 2.2.4 and RStudio Version 1.1.463 on Windows 7. I do not know a lot about using R in general, but I thought a learn lesson would be a good way to check it out. This may be a very simple thing to fix but I simply don't know enough to troubleshoot it.

What could be the problem?

Thanks in advance!

[MarjeanPobuda]

Hi Kayla,

I am sorry to hear that you are running into problems, hopefully I can help. Would you be able to send me what you entered in step 4 of the lesson and what the R console returned to you after entering that code? Step 4 asks that you define the variable 'data_path' by entering the following code:

data_path <- arc.open("C:/African-Buffalo/Ecological Niche Factor Analysis.gdb/ENFA_Environmental_Buffalo_Attributes")

Just as a note though, the location you saved your data at might be different. For example, you might have picked a different folder location and as a result, will need to adjust the path to reflect the folders you used to store the lesson's data in. Additionally, R only recognizes forward slashes ("/") and when you copy a path directly from Windows File Explorer, it contains back slashes ("\") so those need to be adjusted in order for R to know where your data is located. 

I am curious what happened after this step, such as if R threw a warning or error. The reason I ask is because the current error you are seeing says that the variable 'data_path' is not defined. This means that R is says it unaware of any variable called 'data_path' and as a result, is unsure of how to handle your most recent line of code since it uses that variable.

Let me know when you have the chance. Thank you!

[KaylaFlamm]

Marjean,

I figured out that I must have been making a silly mistake in identifying the variable 'data_path'. I see what I was doing wrong there now.

However, I have run into a different problem: Step 1 under the section Produce a habitat suitability map. When I attempt to load the sp package, R asks me to restart because the package is already currently loaded (it apparently loaded when I used library(raster) earlier in the session). If I click Yes to restart R I end up in a loop where R keeps wanting to restart (and interestingly it also gives a Windows message that it has stopped working, though R stays open i.e. does not crash). If I click No or Cancel, I simply cannot load the sp package.

Below is a copy/paste of my RStudio console (at the end I attempted to let R restart twice by clicking Yes, then I clicked No).

> library(arcgisbinding)
*** Please call arc.check_product() to define a desktop license.
> arc.check_product()
product: ArcGIS Pro ( 12.2.0.12813 )
license: Advanced
version: 1.0.1.232
> data_path <- arc.open("G:/ArcGIS_LearnLessons/African-Buffalo/LL Ecological Niche Factor Analysis.gdb/ENFA_Environmental_Buffalo_Attributes")
> install.packages('raster')
trying URL 'https://cran.rstudio.com/bin/windows/contrib/3.5/raster_2.8-4.zip'
Content type 'application/zip' length 3769843 bytes (3.6 MB)
downloaded 3.6 MB

package ‘raster’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
C:\Users\kflamm\AppData\Local\Temp\RtmpM9T3Pa\downloaded_packages
> library(raster)
Loading required package: sp
> arc_raster <- arc.raster(data_path)
> r_raster <- as.raster(arc_raster)
> variables <- as.data.frame(arc_raster$pixel_block())
> variables$pixel_index <- 1:nrow(variables)
> variables <- na.omit(variables)
> pixel_index <- variables$pixel_index
> variables$pixel_index<-NULL
> ecological_variables <-variables[,-56]
> buffalo_presence <- variables[, 56]
> install.packages("ade4")
trying URL 'https://cran.rstudio.com/bin/windows/contrib/3.5/ade4_1.7-13.zip'
Content type 'application/zip' length 5201783 bytes (5.0 MB)
downloaded 5.0 MB

package ‘ade4’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
C:\Users\kflamm\AppData\Local\Temp\RtmpM9T3Pa\downloaded_packages
> library(ade4)
> dudi_obj <- dudi.pca(ecological_variables, scannf = FALSE, nf = 10)
> install.packages("adehabitatHS")
trying URL 'https://cran.rstudio.com/bin/windows/contrib/3.5/adehabitatHS_0.3.13.zip'
Content type 'application/zip' length 1515084 bytes (1.4 MB)
downloaded 1.4 MB

package ‘adehabitatHS’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
C:\Users\kflamm\AppData\Local\Temp\RtmpM9T3Pa\downloaded_packages
> library(adehabitatHS)
Loading required package: adehabitatMA

Attaching package: ‘adehabitatMA’

The following object is masked from ‘package:raster’:

buffer

Loading required package: adehabitatHR
Loading required package: deldir
deldir 0.1-15
Loading required package: adehabitatLT
Loading required package: CircStats
Loading required package: MASS

Attaching package: ‘MASS’

The following objects are masked from ‘package:raster’:

area, select

Loading required package: boot
> enfa_result <- enfa(dudi_obj, buffalo_presence, scannf = FALSE, nf = 2)
> install.packages("sp")
Error in install.packages : Updating loaded packages

Restarting R session...

Loading required package: raster
Loading required package: sp
Error: package or namespace load failed for ‘raster’ in .doLoadActions(where, attach):
error in load action .__A__.1 for package raster: loadModule(module = "spmod", what = TRUE, env = ns, loadNow = TRUE): Unable to load module "spmod": object of type 'closure' is not subsettable
Error in .requirePackage(package) :
unable to find required package ‘raster’
> install.packages("sp")
Error in install.packages : Updating loaded packages

Restarting R session...

Loading required package: raster
Loading required package: sp
Error: package or namespace load failed for ‘raster’ in .doLoadActions(where, attach):
error in load action .__A__.1 for package raster: loadModule(module = "spmod", what = TRUE, env = ns, loadNow = TRUE): Unable to load module "spmod": object of type 'closure' is not subsettable
Error in .requirePackage(package) :
unable to find required package ‘raster’
> install.packages("sp")
Error in install.packages : Updating loaded packages
> install.packages("sp")
Warning in install.packages :
package ‘sp’ is in use and will not be installed

Perhaps I should just reinstall R? I've also noticed it doesn't detect that I am running Pro 2.2.4; instead it seems to think I am running 1.0.1.232 according to the console.

[MarjeanPobuda]

Hi Kayla,

Awesome - I am glad you were able to figure out that first issue. Let's see if we can figure out your new issue. I have run into this problem myself several times and it can be quite frustrating. I think R is struggling in your case because the package sp is required by the package raster. So when R is trying to remove sp it is getting stuck because the raster package is saying, "Hey, don't do that, I need this." Restarting is one way to get around this, but it comes at the cost of losing all of your code and progress so far. Another option is to use the detach function in R. To do this, I believe you would start by actually removing the raster package itself to get rid of the conflict between it and sp. This would look like the following:

detach("package:raster", unload=TRUE)

You can then try doing the same thing for the sp package just in case it is still lingering around:

detach("package:sp", unload=TRUE)

I believe you should now be good to reload the raster package which will automatically come with sp and you should be able to continue on in the lesson. 

library(raster)

Let me know how this works for you or if you run into any other issue. 

Thank you!

-Marjean

[KaylaFlamm]

Detaching the sp package will not work, as it is required by the adehabitateHS package. I went ahead and skipped reloading the sp package and continued onward as follows, successfully creating a habitat suitability plot but at the same time running into another interesting occurrence:

> enfa_result <- enfa(dudi_obj, buffalo_presence, scannf = FALSE, nf = 2)
> # install.packages("sp")  #omitted
> library(sp)
> raster_dim <- dim(arc_raster)[c(2,1)]
> grd <- GridTopology(arc_raster$extent[1:2], arc_raster$cellsize, raster_dim)
> spg <- SpatialPixelsDataFrame(grd, data.frame(d=rep(0, raster_dim[1]*raster_dim[2])))
> spg <- spg[pixel_index,1]
> habitat_suitability <- predict(enfa_result, spg)
Warning messages:
1: In predict.enfa(enfa_result, spg) :
the enfa is not mathematically optimal for prediction:
please consider the madifa instead
2: In points2grid(points, tolerance, round) :
grid has empty column/rows in dimension 1
> plot(habitat_suitability)

My output plot looks nothing like the output example in the Learn Lesson (see image below).

It seems to be missing quite a bit of data, yet the 56 band raster I used for the ecological niche factor analysis model in R (output raster called ENFA_Environmental_Buffalo_Attributes in previous lesson) does indeed have all 56 bands and looks the same as the output example in the previous lesson section, save for the pink bit over the water which I guess might be a different layer still turned on in the learn lesson's example... or I could be wrong.  

[MarjeanPobuda]

Kayla,

Good catch on the adehabitateHS requiring sp. However, if I understand correctly, you did not reload the packages and/or did not restart R/RStudio? It would seem that something has gone astray because you are right, you have lost many of your grid locations and corresponding data. I am not sure what caused this to occur, this is an issue I have not come across before in the course of this lesson. As a first step, I would start by saving your R lines of code into a script file. This way you can easily re-run each line of code without have to retype them. I would then close R and opt to not save your R workspace. Reopen R and rerun all of the lines of code to see if you reproduce the same result with the packages being loaded correctly. If the results are reproduced, can you put your data in a zipped folder and send it to me (mpobuda@esri.com)? I can check on my end if I can reproduce your results as well. 

Thank you and hopefully we will get to the bottom of this soon!

-Marjean

[KaylaFlamm]

You are correct, I cannot reload the sp package or restart R, as quite a few of the loaded packages seem to require the sp package. Therefore I decided to try skipping the step to reload the sp package and simply try calling on its library and going from there. Whether that is the cause of the issue or not I cannot say for certain.

I have emailed you about the data Thank you for looking into this!

[MarkMansell1]

Kayla,

I'm having the same problem as you. Did you ever get an answer or workable solution to missing data problem?

Thanks,

Mark Mansell

[Anonymous User]

Hello Mark and Kayla,

There have been updates to raster tools used in this learn lesson, in particular tools used to create the multi-band rasters. We have redone this learn lesson and it is currently under editing. The updated learn lesson will be out this month.

Also, as of last week we released arcgisbinding version 239. Can you please let me know if the issue with importing sp persists with the new version of the bridge? If so, can you please share the version of R you are using with R-ArcGIS bridge?

Thank you both.

Orhun

[MarkMansell1]

Orhun,

Thanks for getting back to me.

I'll give he course another try when the update comes through.

Mark