I have a number of data tables that I am pulling from a REST API using the GET function in the httr package.
I am having no issues pulling my tables with the following code:
Tables <- c("table1",
"table2",
"table3",
"table4")
for(i in Tables) {
Odata_Request_Variables <- c("Var1","Var2","Var3")
Response <- GET(paste0(`url`,`i`,"?$select=",Odata_Request_Variables),
authenticate(username, password, "ntlm"),
add_headers(.headers = c('Prefer' = 'odata.include-annotations="OData.Community.Display.V1.FormattedValue"')))
DF <- rawToChar(Response$content) %>%
jsonlite::fromJSON(., flatten = TRUE) %>%
as.data.frame(.)
What I get in the end is dataframe called DF.
The issue I am having is that the API limits retrieval of 5000 records and some of my tables exceed that limit. If there are more then 5000 records, a variable is generated in the dataframe called X.odata.nextLink. This contains a url to pass back to the API to retrieve the next 5000 records.
What I need to be able to do is create a loop that looks for the existence of that variable and if it exists to extract the first row (which contains the encoded url), so that I can pass it back into a GET request to retrieve another dataframe. I will then be able to append them together for my complete table.
I am at a loss how to do this, so any help would be greatly appreciated.
Aucun commentaire:
Enregistrer un commentaire