3 Y]e@sLdZddlZddljZddlmZmZddl m Z m Z m Z ddl m Z mZmZddlmZddlmZmZmZdd lmZdd lmZdd lmZmZdd lmZdd lm Z m!Z!ddddgZ"d ddZ#d!ddZ$Gddde e edZ%Gddde%e Z&Gddde%eZ'Gddde edZ(Gddde(ee Z)Gddde(e e Z*dS)"a This module implements multioutput regression and classification. The estimators provided in this module are meta-estimators: they require a base estimator to be provided in their constructor. The meta-estimator extends single output estimators to multioutput estimators. N)ABCMetaabstractmethod) BaseEstimatorcloneMetaEstimatorMixin)RegressorMixinClassifierMixin is_classifier)cross_val_predict) check_array check_X_ycheck_random_state)parallel_helper)if_delegate_has_method)check_is_fittedhas_fit_parameter)check_classification_targets)ParalleldelayedMultiOutputRegressorMultiOutputClassifierClassifierChainRegressorChaincCs2t|}|dk r"|j|||dn |j|||S)N) sample_weight)rfit) estimatorXyrr2lib/python3.6/site-packages/sklearn/multioutput.py_fit_estimator"s  r!TcCsl|r t|}|dk rB|dk r0|j||||dqh|j|||dn&|dk r\|j|||dn |j|||S)N)classesr)r)r")r partial_fit)rrrr"r first_timerrr _partial_fit_estimator+s   r%c@sFeZdZed ddZeddddZdddZd d Zd d Z dS)MultiOutputEstimatorNcCs||_||_dS)N)rn_jobs)selfrr'rrr __init__@szMultiOutputEstimator.__init__rcstddd\jdkr&tddk rDtjd rDtdtd tjd fd d tj dD_ S) aDIncrementally fit the model to data. Fit a separate model for each output variable. Parameters ---------- X : (sparse) array-like, shape (n_samples, n_features) Data. y : (sparse) array-like, shape (n_samples, n_outputs) Multi-output targets. classes : list of numpy arrays, shape (n_outputs) Each array is unique classes for one output in str/int Can be obtained by via ``[np.unique(y[:, i]) for i in range(y.shape[1])]``, where y is the target matrix of the entire dataset. This argument is required for the first call to partial_fit and can be omitted in the subsequent calls. Note that y doesn't need to contain all labels in `classes`. sample_weight : array-like, shape = (n_samples) or None Sample weights. If None, then samples are equally weighted. Only supported if the underlying regressor supports sample weights. Returns ------- self : object T) multi_output accept_sparserzQy must have at least two dimensions for multi-output regression but has only one.Nrz5Underlying estimator does not support sample weights. estimators_)r'c3sP|]H}ttsj|njdd|fdk r>|ndVqdS)N)rr%r,r).0i)rr"r$rr(rrr tsz3MultiOutputEstimator.partial_fit..) r ndim ValueErrorrrhasattrrr'rangeshaper,)r(rrr"rr)rr"r$rr(rr r#Es    z MultiOutputEstimator.partial_fitcstjdstdtddd\tr8tjdkrJtddk rhtjd rhtd tj d fd d t j dD_ S) a Fit the model to data. Fit a separate model for each output variable. Parameters ---------- X : (sparse) array-like, shape (n_samples, n_features) Data. y : (sparse) array-like, shape (n_samples, n_outputs) Multi-output targets. An indicator matrix turns on multilabel estimation. sample_weight : array-like, shape = (n_samples) or None Sample weights. If None, then samples are equally weighted. Only supported if the underlying regressor supports sample weights. Returns ------- self : object rz0The base estimator should implement a fit methodT)r*r+rzQy must have at least two dimensions for multi-output regression but has only one.Nrz5Underlying estimator does not support sample weights.)r'c3s.|]&}ttjdd|fVqdS)N)rr!r)r-r.)rrr(rrr r/sz+MultiOutputEstimator.fit..) r2rr1r r rr0rrr'r3r4r,)r(rrrr)rrr(rr r{s     zMultiOutputEstimator.fitcsVt|dt|jdstdtddt|jdfdd|jD}tj |j S) aPredict multi-output variable using a model trained for each target variable. Parameters ---------- X : (sparse) array-like, shape (n_samples, n_features) Data. Returns ------- y : (sparse) array-like, shape (n_samples, n_outputs) Multi-output targets predicted across multiple predictors. Note: Separate models are generated for each predictor. r,predictz4The base estimator should implement a predict methodT)r+)r'c3s|]}tt|dVqdS)r5N)rr)r-e)rrr r/sz/MultiOutputEstimator.predict..) rr2rr1r rr'r,npZasarrayT)r(rrr)rr r5s      zMultiOutputEstimator.predictcCsddiS)Nmultioutput_onlyTr)r(rrr _more_tagsszMultiOutputEstimator._more_tags)N)NN)N) __name__ __module__ __qualname__rr)rr#rr5r:rrrr r&>s  5 1r&) metaclasscsBeZdZdZd fdd Zedd fdd Zd dd ZZS) raMulti target regression This strategy consists of fitting one regressor per target. This is a simple strategy for extending regressors that do not natively support multi-target regression. Parameters ---------- estimator : estimator object An estimator object implementing `fit` and `predict`. n_jobs : int or None, optional (default=None) The number of jobs to run in parallel for `fit`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. When individual estimators are fast to train or predict using `n_jobs>1` can result in slower performance due to the overhead of spawning processes. Attributes ---------- estimators_ : list of ``n_output`` estimators Estimators used for predictions. Ncstj||dS)N)superr))r(rr') __class__rr r)szMultiOutputRegressor.__init__rcstj|||ddS)aYIncrementally fit the model to data. Fit a separate model for each output variable. Parameters ---------- X : (sparse) array-like, shape (n_samples, n_features) Data. y : (sparse) array-like, shape (n_samples, n_outputs) Multi-output targets. sample_weight : array-like, shape = (n_samples) or None Sample weights. If None, then samples are equally weighted. Only supported if the underlying regressor supports sample weights. Returns ------- self : object )rN)r?r#)r(rrr)r@rr r#sz MultiOutputRegressor.partial_fitcCs"ddlm}|||j||ddS)aEReturns the coefficient of determination R^2 of the prediction. The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the regression sum of squares ((y_true - y_true.mean()) ** 2).sum(). Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0. Notes ----- R^2 is calculated by weighting all the targets equally using `multioutput='uniform_average'`. Parameters ---------- X : array-like, shape (n_samples, n_features) Test samples. y : array-like, shape (n_samples) or (n_samples, n_outputs) True values for X. sample_weight : array-like, shape [n_samples], optional Sample weights. Returns ------- score : float R^2 of self.predict(X) wrt. y. r)r2_scoreZuniform_average)rZ multioutput)ZmetricsrAr5)r(rrrrArrr scores! zMultiOutputRegressor.score)N)N)N) r;r<r=__doc__r)rr#rB __classcell__rr)r@r rs cs:eZdZdZd fdd ZddZddZd d ZZS) ra/Multi target classification This strategy consists of fitting one classifier per target. This is a simple strategy for extending classifiers that do not natively support multi-target classification Parameters ---------- estimator : estimator object An estimator object implementing `fit`, `score` and `predict_proba`. n_jobs : int or None, optional (default=None) The number of jobs to use for the computation. It does each target variable in y in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- estimators_ : list of ``n_output`` estimators Estimators used for predictions. Ncstj||dS)N)r?r))r(rr')r@rr r)DszMultiOutputClassifier.__init__cs>t|dtdd|jDs&tdfdd|jD}|S)amProbability estimates. Returns prediction probabilities for each class of each output. This method will raise a ``ValueError`` if any of the estimators do not have ``predict_proba``. Parameters ---------- X : array-like, shape (n_samples, n_features) Data Returns ------- p : array of shape = [n_samples, n_classes], or a list of n_outputs such arrays if n_outputs > 1. The class probabilities of the input samples. The order of the classes corresponds to that in the attribute `classes_`. r,cSsg|]}t|dqS) predict_proba)r2)r-rrrr [sz7MultiOutputClassifier.predict_proba..z8The base estimator should implement predict_proba methodcsg|]}|jqSr)rE)r-r)rrr rF`s)rallr,r1)r(rZresultsr)rr rEGs    z#MultiOutputClassifier.predict_probacCslt|dt|j}|jdkr&td|jd|krJtdj||jd|j|}tj tj ||kddS)a~Returns the mean accuracy on the given test data and labels. Parameters ---------- X : array-like, shape [n_samples, n_features] Test samples y : array-like, shape [n_samples, n_outputs] True values for X Returns ------- scores : float accuracy_score of self.predict(X) versus y r,rzTy must have at least two dimensions for multi target classification but has only onezCThe number of outputs of Y for fit {0} and score {1} should be same)Zaxis) rlenr,r0r1r4formatr5r7ZmeanrG)r(rrZ n_outputs_Zy_predrrr rBds    zMultiOutputClassifier.scorecCsddiS)N _skip_testTr)r(rrr r:sz MultiOutputClassifier._more_tags)N) r;r<r=rCr)rErBr:rDrr)r@r r+s c@s*eZdZdddZeddZddZdS) _BaseChainNcCs||_||_||_||_dS)N)base_estimatorordercv random_state)r(rLrMrNrOrrr r)sz_BaseChain.__init__c s\t||ddd\}}tj}t|ddj_jdkrTtjt|j d_nNt jt r~jdkr|j |j d_n$t jtt|j dkrtdfdd t|j dD_jdkr|ddjf}tj|rtj||fd d }|j}ntj||f}nbtj|rPtj|j d |j df}tj||fd d }n(tj|j d |j df}tj||f}~xtjD]\}}|ddj|f}|j|ddd|j d|f|jdk r|tjdkr|j d|} tj|ddd| f|jd } tj|rBtj| d|dd| f<n| |dd| f<qWS)aKFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) The input data. Y : array-like, shape (n_samples, n_classes) The target values. Returns ------- self : object T)r*r+)r+NrZrandomz invalid ordercsg|]}tjqSr)rrL)r-_)r(rr rFsz"_BaseChain.fit..Zlil)rIr)rrN)r rrOr rMorder_r7Zarrayr3r4 isinstancestrZ permutationsortedlistr1r,rNspissparsehstackZtocsrZ lil_matrixzeros enumeraterrHr rLZ expand_dims) r(rYrO Y_pred_chainX_aug chain_idxrrZcol_idxZ cv_resultr)r(r rsJ          &   z_BaseChain.fitc Cst|dt|dd}tj|jdt|jf}xvt|jD]h\}}|ddd|f}tj |r|dkrp|}qtj ||f}ntj ||f}|j ||dd|f<q`. Parameters ---------- base_estimator : estimator The base estimator from which the classifier chain is built. order : array-like, shape=[n_outputs] or 'random', optional By default the order will be determined by the order of columns in the label matrix Y.:: order = [0, 1, 2, ..., Y.shape[1] - 1] The order of the chain can be explicitly set by providing a list of integers. For example, for a chain of length 5.:: order = [1, 3, 2, 4, 0] means that the first model in the chain will make predictions for column 1 in the Y matrix, the second model will make predictions for column 3, etc. If order is 'random' a random ordering will be used. cv : int, cross-validation generator or an iterable, optional (default=None) Determines whether to use cross validated predictions or true labels for the results of previous estimators in the chain. If cv is None the true labels are used when fitting. Otherwise possible inputs for cv are: - integer, to specify the number of folds in a (Stratified)KFold, - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. The random number generator is used to generate random chain orders. Attributes ---------- classes_ : list A list of arrays of length ``len(estimators_)`` containing the class labels for each estimator in the chain. estimators_ : list A list of clones of base_estimator. order_ : list The order of labels in the classifier chain. See also -------- RegressorChain: Equivalent for regression MultioutputClassifier: Classifies each output independently rather than chaining. References ---------- Jesse Read, Bernhard Pfahringer, Geoff Holmes, Eibe Frank, "Classifier Chains for Multi-label Classification", 2009. cs(tj||ddt|jD|_|S)aKFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) The input data. Y : array-like, shape (n_samples, n_classes) The target values. Returns ------- self : object cSsg|]\}}|jqSr)classes_)r-r^rrrr rFIsz'ClassifierChain.fit..)r?rrZr,rc)r(rr[)r@rr r:szClassifierChain.fitrLc Cst|dd}tj|jdt|jf}tj|jdt|jf}xt|jD]|\}}|ddd|f}tj|rtj ||f}ntj ||f}|j |dddf|dd|f<|j ||dd|f<qLWtj |j }tjt|j ||j <|dd|f} | S)zPredict probability estimates. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Returns ------- Y_prob : array-like, shape (n_samples, n_classes) T)r+rNr)r r7rYr4rHr,rZrVrWrXrEr5r_rQr`) r(rZ Y_prob_chainr\r^rrar]rbZY_probrrr rENs  " zClassifierChain.predict_probac Cstj|jdt|jf}tj|jdt|jf}x~t|jD]p\}}|ddd|f}tj|rvtj||f}ntj||f}|j ||dd|f<|j ||dd|f<q@Wtj |j }tj t|j ||j <|dd|f} | S)aaEvaluate the decision_function of the models in the chain. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- Y_decision : array-like, shape (n_samples, n_classes ) Returns the decision function of the sample for each model in the chain. rN)r7rYr4rHr,rZrVrWrXdecision_functionr5r_rQr`) r(rZY_decision_chainr\r^rrar]rbZ Y_decisionrrr rdks  z!ClassifierChain.decision_functioncCs dddS)NT)rJr9r)r(rrr r:szClassifierChain._more_tags) r;r<r=rCrrrErdr:rDrr)r@r rs I cs(eZdZdZfddZddZZS)ra+ A multi-label model that arranges regressions into a chain. Each model makes a prediction in the order specified by the chain using all of the available features provided to the model plus the predictions of models that are earlier in the chain. Read more in the :ref:`User Guide `. Parameters ---------- base_estimator : estimator The base estimator from which the classifier chain is built. order : array-like, shape=[n_outputs] or 'random', optional By default the order will be determined by the order of columns in the label matrix Y.:: order = [0, 1, 2, ..., Y.shape[1] - 1] The order of the chain can be explicitly set by providing a list of integers. For example, for a chain of length 5.:: order = [1, 3, 2, 4, 0] means that the first model in the chain will make predictions for column 1 in the Y matrix, the second model will make predictions for column 3, etc. If order is 'random' a random ordering will be used. cv : int, cross-validation generator or an iterable, optional (default=None) Determines whether to use cross validated predictions or true labels for the results of previous estimators in the chain. If cv is None the true labels are used when fitting. Otherwise possible inputs for cv are: - integer, to specify the number of folds in a (Stratified)KFold, - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. The random number generator is used to generate random chain orders. Attributes ---------- estimators_ : list A list of clones of base_estimator. order_ : list The order of labels in the classifier chain. See also -------- ClassifierChain: Equivalent for classification MultioutputRegressor: Learns each output independently rather than chaining. cstj|||S)aKFit the model to data matrix X and targets Y. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) The input data. Y : array-like, shape (n_samples, n_classes) The target values. Returns ------- self : object )r?r)r(rr[)r@rr rszRegressorChain.fitcCsddiS)Nr9Tr)r(rrr r:szRegressorChain._more_tags)r;r<r=rCrr:rDrr)r@r rs@ )N)NNT)+rCZnumpyr7Z scipy.sparseZsparserVabcrrbaserrrrr r Zmodel_selectionr Zutilsr r rZ utils.fixesrZutils.metaestimatorsrZutils.validationrrZutils.multiclassrZ utils._joblibrr__all__r!r%r&rrrKrrrrrr s4        _Zj!