Skip to content

TreeFeaturizer w/ Subsampling #3808

Closed as not planned
Closed as not planned

Description

@justinormont

To support TB+ scale datasets, we should have a row sub-sampling parameter in the Tree Featurizer.

Why TreeFeat? The Tree Featurizer learns the feature interactions on a sub-set of the dataset. The leaves output of Tree Featurizer is then sent to a linear model (SDCA/AP/etc) which is scalable to TB scale datasets.

Gain:
This provides an end-to-end solution for scalable learning. You get the accuracy of the Tree model plus the streamability of the linear model.

Request:

  • Provide a subsample parameter for TreeFeat
  • Inside TreeFeat run the TrainTestSplit to get the correct subsample percent

Other solutions:

  • TrainTest split to create a sub-sample of the dataset before TreeFeat, and train TreeFeat on this subsample. This fails in CV since you need a singular dataview to train on.
  • Shuffle + Take. This fails since our shuffle is block-wise; the 1st 4k rows are shuffled against each other, then the next 4k rows are independently shuffled against each other. This leads to the Take only receiving rows from the top of the file. Also fails in CV.
  • Generate + RangeFilter. We hid our Generate transform. Also fails in CV.

We would like to use the TreeFeat plus a streamable Linear model in AutoML to provide a good model to the user on datasets larger than RAM.

Activity

  1. added and removed on Jan 10, 2020
  2. added
    Priority:2Work that is important, but not critical for the release
    area-TreesTree-based trainers, including FastTree and LightGBM
    and removed on Sep 8, 2026
  3. matouskozak commented on Oct 23, 2026

    @matouskozak
    Member

    Triage: We do not see a high demand for this feature, and we don't have capacity to work on it at the moment. In case this becomes a blocking case for some real-world scenario, please let us know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Priority:2Work that is important, but not critical for the releasearea-TreesTree-based trainers, including FastTree and LightGBMenhancementNew feature or requestyellowmaintenance only

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions