RUXLGBMClassifier

RUXLGBMClassifier represents an approach to machine learning classification tasks, focusing on extracting rules from powerful LGBMClassifier and optimizing them through linear programming for enhanced interpretability.

class ruleopt.RUXLGBMClassifier

A classifier that extracts and optimizes decision rules from a trained LightGBM ensemble model to create a compact and interpretable model. This process involves translating the ensemble’s trees into a set of rules and using optimization to balance model accuracy and interpretability. The complexity of the resulting rule-based model is controlled through a penalty parameter.

__init__(trained_ensemble, *, solver=<ruleopt.solver.highs_solver.HiGHSSolver object>, rule_cost=<ruleopt.rule_cost.rule_cost.Gini object>, class_weight=None, threshold=1e-06, random_state=None)
Parameters:

  • trained_ensemble (lightgbm.LGBMClassifier or lightgbm.Booster) – The trained LightGBM ensemble model from which the rules will be extracted.

  • solver (OptimizationSolver, default=HiGHSSolver()) – An instance of a derived class inherits from the ‘Optimization Solver’ base class. The solver is responsible for optimizing the rule set based on the cost function and constraints.

  • rule_cost (RuleCost or int, default=Gini()) – Defines the cost of rules, either as a specific calculation method (RuleCost instance) or a fixed cost

  • class_weight (dict, "balanced" or None, default=None) – A dictionary mapping class labels to their respective weights, the string “balanced” to automatically adjust weights inversely proportional to class frequencies, or None for no weights. Used to adjust the model in favor of certain classes.

  • threshold (float, default=1.0e-6) – The minimum weight threshold for including a rule in the final model.

  • random_state (int or None, default=None) – Seed for the random number generator to ensure reproducible results.

property classes

Returns unique class labels in the dataset.

Returns:

An array containing the unique class labels of the dataset.

Return type:

np.ndarray

property coefficients

Stores coefficients associated with the rules during optimization.

Returns:

An object or array-like structure storing coefficients related to each rule.

Return type:

Coefficients

property decision_rules

Returns the rules extracted from the decision trees, after optimization.

Returns:

A dictionary where keys are rule indices and values are Rule objects.

Return type:

Dict[int, Rule]

property decision_trees

Returns dictionary that stores the decision tree models.

Returns:

A dictionary containing decision tree models, with identifiers as keys and decision tree instances as values.

Return type:

Dict[int, Any]

fit(x, y, sample_weight=None)

Fits the RUXLGBMClassifier to the training data, optimizing the extracted rules for a balance between accuracy and interpretability.

Parameters:

  • x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

  • y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – The target values (class labels) as integers

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights. If None, then samples equal weights.

Returns:

The fitted model, ready for making predictions.

Return type:

RUXLGBMClassifier

property is_fitted

Indicates whether the model is fitted.

Returns:

True if the model is fitted, False otherwise.

Return type:

bool

property k

Returns the total number of unique classes in the dataset.

Returns:

The total number of unique classes.

Return type:

float

property majority_class

Returns the class label of the majority class in the dataset.

Returns:

The label of the majority class.

Return type:

int

property majority_probability

Returns the probability of the majority class in the dataset.

Returns:

The probability of encountering the majority class in the dataset.

Return type:

float

predict(x, indices=None, threshold=0.0, *, predict_info=False)

Predicts class labels for the given data, optionally returning additional prediction info.

Parameters:

  • x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

  • indices (list or None, default=None) – Specific indices of rules to use for prediction. If None, all rules are used.

  • threshold (float, default=0) – The threshold for selecting rules based on their weights.

  • predict_info (bool, default=False) – If True, returns additional information about the prediction process including indices of samples with missed values, number of rules applied per sample, and average rule length per sample. Otherwise, returns only the predicted class labels.

Returns:

An array of predicted class labels for each instance in x. If predict_info is True, also returns arrays containing indices of samples with missed values, number of rules applied per sample, and average rule length per sample.

Return type:

np.ndarray

predict_proba(x, indices=None, threshold=0.0, *, predict_info=False)

Predicts class probabilities for the given data, optionally returning additional prediction info.

Parameters:

  • x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

  • indices (list or None, default=None) – Specific indices of rules to use for calculating probabilities. If None, all rules are used.

  • threshold (float, default=0) – The threshold for selecting rules based on their weights.

  • predict_info (bool, default=False) – If True, returns additional information about the prediction process including indices of samples with missed values, number of rules applied per sample, and average rule length per sample. Otherwise, returns only the probabilities of each class for each sample.

Returns:

An array where each row corresponds to a sample in x and each column to a class, containing the probability of each class for each sample. If predict_info is True, also returns arrays containing indices of samples with missed values, number of rules applied per sample, and average rule length per sample.

Return type:

np.ndarray

property rule_columns

Returns indices of rules selected as part of the model.

Returns:

An array of indices corresponding to the rules included in the model.

Return type:

np.ndarray

property rule_info

Returns information about each rule.

Returns:

A dictionary with rule indices as keys and tuples containing information about each rule as values. The tuple structure is (rule_id, feature_index, threshold, values_array).

Return type:

Dict[int, Tuple[int, int, int, np.ndarray]]