RUXClassifier

RUXClassifier represents an approach to machine learning classification tasks, focusing on extracting rules from powerful sklearn ensemble models and optimizing them through linear programming for enhanced interpretability.

class ruleopt.RUXClassifier

RUXClassifier aims to build a compact and interpretable model by employing rule-based learning extracted from a trained scikit-learn ensemble model such as Random Forests, Gradient Boosting Machines, and Extra-Trees Classifiers.

__init__(trained_ensemble, *, solver=<ruleopt.solver.highs_solver.HiGHSSolver object>, rule_cost=<ruleopt.rule_cost.rule_cost.Gini object>, class_weight=None, threshold=1e-06, random_state=None)

Parameters:

• trained_ensemble (sklearn ensemble) – A fitted scikit-learn ensemble model such as RandomForestClassifier, GradientBoostingClassifier, or ExtraTreesClassifier.

• solver (OptimizationSolver, default=HiGHSSolver()) – An instance of a derived class inherits from the ‘Optimization Solver’ base class.

• rule_cost (RuleCost or int, default=Gini()) – Defines the cost of rules, either as a specific calculation method (RuleCost instance) or a fixed cost.

• class_weight (dict, "balanced" or None, default=None) – A dictionary mapping class labels to their respective weights, the string “balanced” to automatically adjust weights inversely proportional to class frequencies, or None for no weights.

• threshold (float, default=1.0e-6) – The minimum weight threshold for including a rule in the final model.

• random_state (int or None, default=None) – Seed for the random number generator to ensure reproducible results.

fit(x, y, sample_weight=None)

Fits the RUXClassifier by extracting rules from the trained ensemble and optimizing their weights.

Parameters:

• x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

• y (array-like of shape (n_samples,)) – The target values (class labels) as integers.

• sample_weight (array-like of shape (n_samples,), default=None) – Sample weights. If None, then samples get equal weights.

Returns:

The fitted model, ready for making predictions.

Return type:

RUXClassifier

property classes

Returns unique class labels in the dataset.

Returns:

An array containing the unique class labels of the dataset.

Return type:

np.ndarray

property coefficients

Stores coefficients associated with the rules during optimization.

Returns:

An object or array-like structure storing coefficients related to each rule.

Return type:

Coefficients

property decision_rules

Returns the rules extracted from the decision trees, after optimization.

Returns:

A dictionary where keys are rule indices and values are Rule objects.

Return type:

Dict[int, Rule]

property decision_trees

Returns dictionary that stores the decision tree models.

Returns:

A dictionary containing decision tree models, with identifiers as keys and decision tree instances as values.

Return type:

Dict[int, Any]

property is_fitted

Indicates whether the model is fitted.

Returns:

True if the model is fitted, False otherwise.

Return type:

bool

property k

Returns the total number of unique classes in the dataset.

Returns:

The total number of unique classes.

Return type:

float

property majority_class

Returns the class label of the majority class in the dataset.

Returns:

The label of the majority class.

Return type:

int

property majority_probability

Returns the probability of the majority class in the dataset.

Returns:

The probability of encountering the majority class in the dataset.

Return type:

float

predict(x, indices=None, threshold=0.0, *, predict_info=False)

Predicts class labels for the given data, optionally returning additional prediction info.

Parameters:

• x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

• indices (list or None, default=None) – Specific indices of rules to use for prediction. If None, all rules are used.

• threshold (float, default=0) – The threshold for selecting rules based on their weights.

• predict_info (bool, default=False) – If True, returns additional information about the prediction process including indices of samples with missed values, number of rules applied per sample, and average rule length per sample. Otherwise, returns only the predicted class labels.

Returns:

An array of predicted class labels for each instance in x. If predict_info is True, also returns arrays containing indices of samples with missed values, number of rules applied per sample, and average rule length per sample.

Return type:

np.ndarray

predict_proba(x, indices=None, threshold=0.0, *, predict_info=False)

Predicts class probabilities for the given data, optionally returning additional prediction info.

Parameters:

• x (array-like of shape (n_samples, n_features)) – The training input samples. Internally, it will be converted to dtype=np.float32.

• indices (list or None, default=None) – Specific indices of rules to use for calculating probabilities. If None, all rules are used.

• threshold (float, default=0) – The threshold for selecting rules based on their weights.

• predict_info (bool, default=False) – If True, returns additional information about the prediction process including indices of samples with missed values, number of rules applied per sample, and average rule length per sample. Otherwise, returns only the probabilities of each class for each sample.

Returns:

An array where each row corresponds to a sample in x and each column to a class, containing the probability of each class for each sample. If predict_info is True, also returns arrays containing indices of samples with missed values, number of rules applied per sample, and average rule length per sample.

Return type:

np.ndarray

property rule_columns

Returns indices of rules selected as part of the model.

Returns:

An array of indices corresponding to the rules included in the model.

Return type:

np.ndarray

property rule_info

Returns information about each rule.

Returns:

A dictionary with rule indices as keys and tuples containing information about each rule as values. The tuple structure is (rule_id, feature_index, threshold, values_array).

Return type:

Dict[int, Tuple[int, int, int, np.ndarray]]