Does FSelectorRcpp produce the same results as FSelector?

[larskotthoff]

Do you guys have any tests to check this? We're thinking of replacing FSelector with FSelectorRcpp in mlr, but we'd like to be sure that we remain reproducible.

@berndbischl

[zzawadz]

For some functions - yes - see e.g. https://github.com/mi2-warsaw/FSelectorRcpp/blob/master/tests/testthat/test-information_gain.R

This is part of the code:

test_that("Comparsion with FSelector", {
  expect_equal(information.gain(Species ~ ., data = iris)$attr_importance,
               information_gain(formula = Species ~ ., data = iris)$importance)

  expect_equal(gain.ratio(Species ~ ., data = iris)$attr_importance,
               information_gain(formula = Species ~ ., data = iris,
                                type = "gainratio")$importance)

  expect_equal(symmetrical.uncertainty(Species ~ .,
                                       data = iris)$attr_importance,
               information_gain(formula = Species ~ ., data = iris,
                                type = "symuncert")$importance)
})

For other functions please send us a list of functionalities which must be checked against FSelector, and then we will prepare required tests to convince you that everything is fine:)

[larskotthoff]

I'd love to see tests for all of the functions that users can call, ideally on a range of different inputs. Maybe using quickcheck (https://github.com/RevolutionAnalytics/quickcheck)?

[larskotthoff]

Oh and once I'm convinced I'm willing to officially deprecate FSelector in favour of FSelectorRcpp.

[zzawadz]

Ok. We will work on this.

Thanks!

[MarcinKosinski]

@zzawadz another amazing challenge for FSelectorRcpp : )

Maybe it'll be the easiest way to include FSelectorRcpp in the FSelector

[zzawadz]

@MarcinKosinski
Good idea. We can replace functionalities (inner implementation) in FSelector step by step to reach the convergence. @larskotthoff What do you think?

[larskotthoff]

Sounds good. Pull requests welcome!

[MarcinKosinski]

So this can be closed - #27 : ) @larskotthoff is aware of that we will suggest inner implementation

[MarcinKosinski]

Getting back to this thread.
FSelectorRcpp will be available on CRAN again soon (removed because lack of informtion of C++ dependency) #69

To enable FSelectorRcpp be a part of FSelector engine I think we could try
substituting

FSelector:::information.gain.body() function with the FSelectorRcpp::information_gain(). We need to polish FSelectorRcpp edition to produce the same results as FSelector and also enable some another approaches to dealing with NAs and discretization of dependent variable.

2 tasks should be finished then

• Enable dependent variable discretization the same as FSelector:::equal.frequency.binning.discretization #62 Enable dependent variable discretization the same as FSelector:::equal.frequency.binning.discretization - FSelectorRcpp does not provide discretization for the dependent variable. To make it suitable with FSelector we will enable extra option to discretize the dependent variable (

FSelector:::information.gain.body <- function(params, equal = TRUE) {
    FSelectorRcpp::information_gain(params, equal = equal)
}

• Enable FSelectoRcpp dealing with NAs in explanatory variables as in the RWeka::Discretize  #63 Enable FSelectoRcpp dealing with NAs in explanatory variables as in the RWeka::Discretize we slightly need to reorganize the code, so that we only remove rows that have NAs in the dependent variable (and not in any variable considered to be discretized as it was done before) and that we can provide the exact same explanatory variable discretization as in RWeka::Discretize

[RandomGuessR]

Hi, I am struggling to get the same results from FSelectorRcpp and FSelector - posted under this issue: mlr-org/mlr#1677 (comment).
The results I get are actually very different, and the impact on an end model is large.
Would appreciate your help if I am doing anything wrong.
Thanks!