I am trying to automatize a process in R that we were doing with another program. My objective is to clean a large data frame (lets call it "RData"), and create a column at the end called "Interp" for interpretation. The interpretation column has different values based on several variables and combinations. For the sake of the argument I will limit the variables to 6 tests ("apple1", "apple2", "banana1", "banana2", "pear1", "pear2"). The "Interp" column has 7 possible outcomes ("PosA", "PosP", "PosPA", " Neg", "Pending", "DUO" and "Other"). The 6 Tests have 4 Possible outcomes ("P", "N", "E" and "NA"). It would look something like this:
"ID" "apple1", "apple2", "banana1", "banana2", "pear1", "pear2" "Interp"
1 N NA 1 A N P "Other"
2 P N 0 B P P "PosPA"
3 NA NA 1 C NA P "Other"
4 N NA 0 A N N "Neg"
5 N NA 1 B N P "Other"
6 NA NA 0 C N N "Other"
In order to do this I created a function that first cleans the row on a variable. It just eliminates undesired rows (I don't believe it's relevant, but I rather mention it just in case), and then runs a long ifelse algorithm to get the desired results on the "Interp" column based on the results from the 6 tests. I have a control data frame (lets call it "CData") to make sure the code is running correctly. "CData" showed me that some of my results are being classified as "Other" when they should be classified as a type of "Pos".
This is where it becomes interesting. I took out the problem rows and placed them in a much smaller data frame (lets call it "RsData") to better deal with the "trouble rows". When I ran the identical ifelse algorithm again it worked. The rows obtained the correct type value of "Pos" in the "Interp" column. It made it seem like the algorithm worked on a smaller data frame (about 5 rows), but not when it was large (in the thousands). The problem is that my code is running without error, but the ifesle algorithm gives me an incorrect results for some rows when working on the large data frame, but gives me the correct values for those same rows after they are placed in a smaller data frame. This is an example of my results with the smaller data frame:
"ID" "apple1", "apple2", "banana1", "banana2", "pear1", "pear2" "Interp"
1 N NA 1 A N P "PosP"
3 NA NA 1 C NA P "PosP"
5 N NA 1 B N P "PosP"
I found a pattern in the specific rows that we coming out wrong. I noticed that if the row had a value of NA in "apple2" its results in "Interp" would be "Other". I noticed this while going deep into the data, but I am almost sure I covered this in my algorithm. Specifically in this line ifelse(RData$pear2 =="P" & is.na(RData$apple2) & RData$banana1 == 1,"PosP",... , but that's the only thing that I have noticed.
In short I need help in understanding why my algorithm is not working in the big data set. I would guess it's a mistake in my coding, but it works perfectly when I isolate a sample of the problem rows in a smaller data set.
This is the ifelse algorithm I am using if you would like to view it:
ifelse(RData$pear1 == "P" & RData$apple1 =="P", "PosPA",
ifelse(RData$pear1 == "P", "PosP",
ifelse(RData$apple1 == "P", "PosA",
ifelse(RData$pear2 =="P" & is.na(RData$apple2) & RData$banana1 == 1,"PosP",
ifelse(RData$pear2 == "P" & RData$apple2 =="E" & RData$banana1 == 1, "PosP",
ifelse(RData$pear2 == "P" & RData$apple2 =="P" & RData$banana1 == 1, "PosP",
ifelse(is.na(RData$pear2) & RData$apple2 =="P" & RData$banana1 == 1, "PENDING",
ifelse(RData$pear2 == "P" & RData$apple2 =="N", "PosP",
ifelse(RData$pear2 == "P" & RData$apple2 =="P", "DUO",
ifelse(RData$pear2 == "P" & RData$apple2 =="E", "DUO",
ifelse(RData$pear2 =="P" & is.na(RData$apple2), "PENDING",
ifelse(RData$pear2 == "P" & is.na(RData$apple2) & RData$banana2 =="A", "PosP", ##Temp##
ifelse(RData$pear2 == "N" & RData$apple2 =="P", "PosA",
ifelse(RData$pear2 == "E" & RData$apple2 =="P", "DUO",
ifelse(is.na(RData$pear2) & RData$apple2 =="P", "DUO",##3##
ifelse(RData$pear1 == "N" & RData$apple2 =="P" & RData$banana2 =="B", "PosA", ##4##
ifelse(RData$pear1 == "N" & RData$apple2 =="P" & RData$banana2 =="C", "PosA", ##5##
ifelse(RData$pear1 == "N" & is.na(RData$pear2) & RData$banana2 =="B", "Neg",
ifelse(RData$pear1 == "N" & is.na(RData$pear2) & RData$banana2 =="D", "Neg",
ifelse(RData$pear1 == "N" & is.na(RData$pear2) & RData$banana2 =="C", "Neg",
ifelse(RData$pear1 == "N" & is.na(RData$pear2) & RData$banana2 =="F", "Pending",
ifelse(is.na(RData$pear1) & is.na(RData$pear2) & RData$banana2 =="F", "Pending", "Other))))))))))))))))))))))))
Aucun commentaire:
Enregistrer un commentaire