.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/getting_started/plot_getting_started.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_getting_started_plot_getting_started.py: .. _example_getting_started: ====================== Skore: getting started ====================== This guide illustrates how to use skore through a complete machine learning workflow for binary classification: #. Set up a proper experiment with training and test data #. Develop and evaluate multiple models using cross-validation #. Compare models to select the best one #. Validate the final model on held-out data #. Track and organize your machine learning results Throughout this guide, we will see how skore helps you: * Avoid common pitfalls with smart diagnostics * Quickly get rich insights into model performance * Organize and track your experiments Storing reports in Skore Hub ---------------------------- At the end of this example, we send the reports in Skore Hub (https://skore.probabl.ai/) that is a platform for storing, sharing and exploring your machine learning reports. To run this example and push in your own Skore Hub workspace and project, you can run this example with the following command: .. code-block:: bash WORKSPACE= PROJECT= python plot_getting_started.py In this gallery, we are going to push the different reports into a public workspace. .. GENERATED FROM PYTHON SOURCE LINES 42-47 Setting up our classification problem ===================================== Let's start by loading the "toxicity" dataset, a classification problem where we classify tweets as "toxic" or "not toxic". .. GENERATED FROM PYTHON SOURCE LINES 49-55 .. code-block:: Python from skrub import TableReport, datasets toxicity = datasets.fetch_toxicity() X, y = toxicity.X, toxicity.y TableReport(toxicity.toxicity) .. rst-class:: sphx-glr-script-out .. code-block:: none Downloading 'toxicity_v1' from https://github.com/skrub-data/skrub-data-files/raw/refs/heads/main/toxicity_v1.zip (attempt 1/3) .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 56-58 We create a held-out test set to evaluate our final model once we are done experimenting. .. GENERATED FROM PYTHON SOURCE LINES 60-66 .. code-block:: Python from sklearn.model_selection import train_test_split X_experiment, X_holdout, y_experiment, y_holdout = train_test_split( X, y, random_state=0 ) .. GENERATED FROM PYTHON SOURCE LINES 67-81 Model development with cross-validation ======================================= We will investigate two different families of models using cross-validation. 1. A :class:`~sklearn.linear_model.LogisticRegression` which is a linear model 2. A :class:`~sklearn.ensemble.RandomForestClassifier` which is a more powerful model. In both cases, we rely on :func:`skrub.tabular_pipeline` to choose the proper preprocessing depending on the kind of model. Cross-validation is necessary to get a more reliable estimate of model performance. skore makes it easy through :class:`skore.CrossValidationReport`. .. GENERATED FROM PYTHON SOURCE LINES 83-89 Model no. 1: logistic regression with preprocessing --------------------------------------------------- Our first model will be a linear model, with automatic preprocessing of the text feature. Under the hood, skrub's :class:`~skrub.TableVectorizer` will adapt the preprocessing based on our choice to use a linear model. .. GENERATED FROM PYTHON SOURCE LINES 91-97 .. code-block:: Python from sklearn.linear_model import LogisticRegression from skrub import tabular_pipeline logistic_regression = tabular_pipeline(LogisticRegression()) logistic_regression .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 98-102 We now evaluate our model with cross-validation, using :func:`~skore.evaluate` with `splitter=5` to perform 5-fold cross-validation. This returns a :class:`~skore.CrossValidationReport` object, which can be used to access the performance metrics and other information about the model. .. GENERATED FROM PYTHON SOURCE LINES 104-111 .. code-block:: Python from skore import evaluate logreg_cv_report = evaluate( logistic_regression, X_experiment, y_experiment, pos_label="Toxic", splitter=5 ) logreg_cv_report .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 112-120 A report will quickly show important information regarding the performance of the model, the dataset used and the architecture of the model. This information is only a quick overview and one can dig deeper into the report to get more information. Indeed, Skore reports allow to structure the statistical information we look for when experimenting with predictive models. First, the :meth:`~skore.CrossValidationReport.help` method shows us all its available methods and attributes, with the knowledge that our model was trained for classification: .. GENERATED FROM PYTHON SOURCE LINES 122-124 .. code-block:: Python logreg_cv_report.help() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 125-126 For example, we can examine the training data, which excludes the held-out data: .. GENERATED FROM PYTHON SOURCE LINES 128-130 .. code-block:: Python logreg_cv_report.data.summarize() .. raw:: html

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 131-132 Additionally we can run automatic checks on the model and get a summary of the findings: .. GENERATED FROM PYTHON SOURCE LINES 134-136 .. code-block:: Python logreg_cv_report.checks.summarize() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 137-139 But we can also quickly get an overview of the performance of our model, using :meth:`~skore.CrossValidationReport.metrics.summarize`: .. GENERATED FROM PYTHON SOURCE LINES 141-144 .. code-block:: Python logreg_metrics = logreg_cv_report.metrics.summarize() logreg_metrics.frame(favorability=True) .. raw:: html
logisticregression_mean logisticregression_std favorability
metric
accuracy 0.822667 0.032180 (↗︎)
precision 0.823620 0.035558 (↗︎)
recall 0.825368 0.036734 (↗︎)
roc_auc 0.901039 0.033199 (↗︎)
log_loss 0.393744 0.054369 (↘︎)
brier_score 0.126894 0.021050 (↘︎)
fit_time 0.182098 0.003279 (↘︎)
predict_time 0.032289 0.000350 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 145-149 .. note:: `favorability=True` adds a column showing whether higher or lower metric values are better. .. GENERATED FROM PYTHON SOURCE LINES 151-153 In addition to the summary of metrics, skore provides more advanced statistical information such as the precision-recall curve: .. GENERATED FROM PYTHON SOURCE LINES 155-158 .. code-block:: Python precision_recall = logreg_cv_report.metrics.precision_recall() precision_recall.help() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 159-164 .. note:: The output of :meth:`~skore.CrossValidationReport.metrics.precision_recall` is a :class:`~skore.Display` object. This is a common pattern in skore which allows us to access the information in several ways. .. GENERATED FROM PYTHON SOURCE LINES 166-167 We can visualize the critical information as a plot, with only a few lines of code: .. GENERATED FROM PYTHON SOURCE LINES 169-171 .. code-block:: Python _ = precision_recall.plot() .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_001.png :alt: Precision-Recall Curve for LogisticRegression Positive label: Toxic Data source: Test set :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 172-173 Or we can access the raw information as a dataframe if additional analysis is needed: .. GENERATED FROM PYTHON SOURCE LINES 175-177 .. code-block:: Python precision_recall.frame() .. raw:: html
split threshold precision recall
0 0 0.004611 0.506667 1.000000
1 0 0.023608 0.531469 1.000000
2 0 0.023939 0.528169 0.986842
3 0 0.066300 0.576923 0.986842
4 0 0.070432 0.573643 0.973684
... ... ... ... ...
468 4 0.995320 1.000000 0.066667
469 4 0.996643 1.000000 0.053333
470 4 0.996861 1.000000 0.040000
471 4 0.999029 1.000000 0.026667
472 4 0.999881 1.000000 0.013333

473 rows × 4 columns



.. GENERATED FROM PYTHON SOURCE LINES 178-179 As another example, we can plot the confusion matrix with the same consistent API: .. GENERATED FROM PYTHON SOURCE LINES 181-184 .. code-block:: Python confusion_matrix = logreg_cv_report.metrics.confusion_matrix() _ = confusion_matrix.plot() .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_002.png :alt: Confusion Matrix Data source: Test set :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 185-187 Skore also provides utilities to inspect models. Since our model is a linear model, we can study the importance that it gives to each feature: .. GENERATED FROM PYTHON SOURCE LINES 189-192 .. code-block:: Python coefficients = logreg_cv_report.inspection.coefficients() coefficients.frame() .. raw:: html
feature coefficient_mean coefficient_std
0 Intercept -0.243448 0.087540
1 text_00 0.520481 0.072374
2 text_01 0.640356 0.170732
3 text_02 0.006669 2.267251
4 text_03 0.255316 1.056051
5 text_04 1.380711 0.995806
6 text_05 -0.243086 0.823388
7 text_06 0.353746 0.362825
8 text_07 0.204637 0.244194
9 text_08 -0.035371 0.733832
10 text_09 0.376660 0.610687
11 text_10 -0.201819 0.511644
12 text_11 -0.099357 0.205994
13 text_12 -0.079186 0.527071
14 text_13 -0.055234 0.338381
15 text_14 0.002133 0.403027
16 text_15 -0.010264 0.239392
17 text_16 0.204061 0.387195
18 text_17 -0.194062 0.266098
19 text_18 0.176111 0.546032
20 text_19 -0.056368 0.297341
21 text_20 -0.377464 0.313493
22 text_21 -0.103353 0.292703
23 text_22 -0.100379 0.062667
24 text_23 0.000174 0.170993
25 text_24 -0.130224 0.235263
26 text_25 -0.307009 0.360734
27 text_26 -0.173430 0.213544
28 text_27 -0.154933 0.085061
29 text_28 0.180420 0.356470
30 text_29 0.149520 0.180058


.. GENERATED FROM PYTHON SOURCE LINES 193-195 .. code-block:: Python _ = coefficients.plot(select_k=15) .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_003.png :alt: Coefficients of LogisticRegression :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 196-203 Model no. 2: Random forest -------------------------- Now, we cross-validate a more powerful model using :class:`~sklearn.ensemble.RandomForestClassifier`. Again, we rely on :func:`~skrub.tabular_pipeline` to perform the appropriate preprocessing to use with this model. .. GENERATED FROM PYTHON SOURCE LINES 205-210 .. code-block:: Python from sklearn.ensemble import RandomForestClassifier random_forest = tabular_pipeline(RandomForestClassifier(random_state=0)) random_forest .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 211-216 .. code-block:: Python rf_cv_report = evaluate( random_forest, X_experiment, y_experiment, pos_label="Toxic", splitter=5 ) rf_cv_report .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 217-219 .. code-block:: Python rf_cv_report.checks.summarize() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 220-221 We will now compare this new model with the previous one. .. GENERATED FROM PYTHON SOURCE LINES 223-229 Comparing our models ==================== Now that we have our two models, we need to decide which one should go into production. We can compare them with the :func:`~skore.compare` function that returns a :class:`~skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 231-241 .. code-block:: Python from skore import compare comparison = compare( { "logistic regression": logreg_cv_report, "random forest": rf_cv_report, }, ) comparison .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 242-243 This report follows the same API as :class:`~skore.CrossValidationReport`: .. GENERATED FROM PYTHON SOURCE LINES 243-245 .. code-block:: Python comparison.help() .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 246-248 We have access to the same tools to perform statistical analysis and compare both models: .. GENERATED FROM PYTHON SOURCE LINES 248-251 .. code-block:: Python comparison_metrics = comparison.metrics.summarize() comparison_metrics.frame(favorability=True) .. raw:: html
mean_logistic_regression mean_random_forest std_logistic_regression std_random_forest favorability
metric
accuracy 0.822667 0.812000 0.032180 0.031411 (↗︎)
precision 0.823620 0.841484 0.035558 0.048526 (↗︎)
recall 0.825368 0.775088 0.036734 0.044436 (↗︎)
roc_auc 0.901039 0.880024 0.033199 0.043085 (↗︎)
log_loss 0.393744 0.458317 0.054369 0.044647 (↘︎)
brier_score 0.126894 0.145625 0.021050 0.018738 (↘︎)
fit_time 0.182098 0.399920 0.003279 0.004154 (↘︎)
predict_time 0.032289 0.041152 0.000350 0.000471 (↘︎)


.. GENERATED FROM PYTHON SOURCE LINES 252-254 .. code-block:: Python _ = comparison.metrics.precision_recall().plot() .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_004.png :alt: Precision-Recall Curve Positive label: Toxic Data source: Test set, estimator = logistic regression, estimator = random forest :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 255-260 Based on the previous tables and plots, it seems that the :class:`~sklearn.ensemble.RandomForestClassifier` model has slightly worse performance due to overfitting on this small dataset. We make the choice to deploy the linear model to make a comparison with the coefficients study shown earlier. .. GENERATED FROM PYTHON SOURCE LINES 262-269 Final model evaluation on held-out data ======================================= Now that we have chosen to deploy the linear model, we will train it on the full experiment set and evaluate it on our held-out data: training on more data should help performance and we can also validate that our model generalizes well to new data. This can be done in one step with :meth:`~skore.ComparisonReport.create_estimator_report`. .. GENERATED FROM PYTHON SOURCE LINES 271-277 .. code-block:: Python final_report = comparison.create_estimator_report( report_key="logistic regression", X_test=X_holdout, y_test=y_holdout ) final_report .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 278-280 This returns a :class:`~skore.EstimatorReport` which has a similar API to the other report classes: .. GENERATED FROM PYTHON SOURCE LINES 282-285 .. code-block:: Python final_metrics = final_report.metrics.summarize() final_metrics.frame() .. rst-class:: sphx-glr-script-out .. code-block:: none metric accuracy 0.816000 precision 0.834783 recall 0.780488 roc_auc 0.912105 log_loss 0.368210 brier_score 0.118757 fit_time 0.183366 predict_time 0.042362 Name: LogisticRegression, dtype: float64 .. GENERATED FROM PYTHON SOURCE LINES 286-288 .. code-block:: Python _ = final_report.metrics.confusion_matrix().plot() .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_005.png :alt: Confusion Matrix Data source: Test set :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 289-293 We can easily combine the results of the previous cross-validation together with the evaluation on the held-out dataset, since the two are accessible as tables. This way, we can check if our chosen model meets the expectations we set during the experiment phase. .. GENERATED FROM PYTHON SOURCE LINES 295-299 .. code-block:: Python final_frame = final_metrics.frame().to_frame() cv_frame = logreg_cv_report.metrics.summarize().frame() final_frame.merge(cv_frame, on="metric", how="outer") .. raw:: html
LogisticRegression logisticregression_mean logisticregression_std
metric
accuracy 0.816000 0.822667 0.032180
brier_score 0.118757 0.126894 0.021050
fit_time 0.183366 0.182098 0.003279
log_loss 0.368210 0.393744 0.054369
precision 0.834783 0.823620 0.035558
predict_time 0.042362 0.032289 0.000350
recall 0.780488 0.825368 0.036734
roc_auc 0.912105 0.901039 0.033199


.. GENERATED FROM PYTHON SOURCE LINES 300-302 As expected, our final model gets better performance, likely thanks to the larger training set. .. GENERATED FROM PYTHON SOURCE LINES 304-306 Our final sanity check is to compare the features considered most impactful between our final model and the cross-validation: .. GENERATED FROM PYTHON SOURCE LINES 308-324 .. code-block:: Python final_coefficients = final_report.inspection.coefficients() cv_coefficients = logreg_cv_report.inspection.coefficients() features_final_coefficients = final_coefficients.frame(select_k=15)["feature"] features_cv_coefficients = cv_coefficients.frame(select_k=15)["feature"] print( f"Most important features available in both models: " f"{set(features_final_coefficients).intersection(set(features_cv_coefficients))}" ) print( f"Most important features available in final model but not in cross-validation: " f"{set(features_final_coefficients).difference(set(features_cv_coefficients))}" ) .. rst-class:: sphx-glr-script-out .. code-block:: none Most important features available in both models: {'text_09', 'text_16', 'text_25', 'text_00', 'text_04', 'text_06', 'text_01', 'text_02', 'text_10', 'text_12'} Most important features available in final model but not in cross-validation: {'text_24', 'text_14', 'Intercept', 'text_07', 'text_28'} .. GENERATED FROM PYTHON SOURCE LINES 325-327 We can further check if there is a drastic difference in the ordering by plotting those features with the largest absolute coefficients. .. GENERATED FROM PYTHON SOURCE LINES 329-332 .. code-block:: Python final_coefficients.plot(select_k=15, sorting_order="descending") _ = cv_coefficients.plot(select_k=15, sorting_order="descending") .. rst-class:: sphx-glr-horizontal * .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_006.png :alt: Coefficients of LogisticRegression :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_006.png :class: sphx-glr-multi-img * .. image-sg:: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_007.png :alt: Coefficients of LogisticRegression :srcset: /auto_examples/getting_started/images/sphx_glr_plot_getting_started_007.png :class: sphx-glr-multi-img .. GENERATED FROM PYTHON SOURCE LINES 333-334 They seem very similar, so we are done! .. GENERATED FROM PYTHON SOURCE LINES 336-356 Tracking our work with a skore Project ====================================== Now that we have completed our modeling workflow, we should store our models in a safe place for future work. Indeed, if this research notebook were modified, we would no longer be able to relate the current production model to the code that generated it. We can use a :class:`skore.Project` to keep track of our experiments. This makes it easy to organize, retrieve, and compare models over time. Usually this would be done as you go along the model development, but in the interest of simplicity we kept this until the end. We are using Skore Hub (https://skore.probabl.ai/) to store and review our reports. .. note:: Here, we are using Skore Hub to store and analyze the reports that we computed. Note that you can store reports as well locally using `mode="local"` when creating or loading projects via `skore.Project`. .. GENERATED FROM PYTHON SOURCE LINES 356-363 .. code-block:: Python from skore import login login() .. rst-class:: sphx-glr-script-out .. code-block:: none ╭───────────────────────────────── Login to Skore Hub ─────────────────────────────────╮ │ │ │ Successfully logged in, using API key. │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ .. GENERATED FROM PYTHON SOURCE LINES 398-399 We load or create a hub project: .. GENERATED FROM PYTHON SOURCE LINES 399-402 .. code-block:: Python project = Project(name=PROJECT, mode="hub", workspace=WORKSPACE) .. GENERATED FROM PYTHON SOURCE LINES 403-404 We store our reports with descriptive keys: .. GENERATED FROM PYTHON SOURCE LINES 404-407 .. code-block:: Python project.put("logreg_cv", logreg_cv_report) .. rst-class:: sphx-glr-script-out .. code-block:: none /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() Putting logreg_cv 0:01:35 Consult your report at https://skore.probabl.ai/skore/example-getting-started-pull-3233/cross-validations/38930 .. GENERATED FROM PYTHON SOURCE LINES 408-410 .. code-block:: Python project.put("rf_cv", rf_cv_report) .. rst-class:: sphx-glr-script-out .. code-block:: none /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() /home/runner/work/skore/skore/skore/venv/lib/python3.14/site-packages/skore/_plugins/hub/artifact/environment.py:48: UserWarning: Package matplotlib_skore_scraper seems to be an editable or local install (loaded from outside site-packages). It will not be recorded in the inferred requirements. for requirement in infer() Putting rf_cv 0:01:26 Consult your report at https://skore.probabl.ai/skore/example-getting-started-pull-3233/cross-validations/38936 .. GENERATED FROM PYTHON SOURCE LINES 411-413 In this example, we created a read-only Skore Hub project that you can visit by clicking on the link above and explore the reports. .. GENERATED FROM PYTHON SOURCE LINES 415-416 Now we can retrieve a summary of our stored reports: .. GENERATED FROM PYTHON SOURCE LINES 418-421 .. code-block:: Python summary = project.summarize() summary .. raw:: html


.. GENERATED FROM PYTHON SOURCE LINES 422-428 .. note:: :meth:`~skore.Project.summarize` returns a :class:`~skore.Summary` object. In a Jupyter environment it renders as an interactive table where you can filter rows and pick reports across the different views; the selection produces a query string ready to pass to :meth:`~skore.Summary.query` so you can recover exactly those reports. .. GENERATED FROM PYTHON SOURCE LINES 430-434 Once you filtered the summary (e.g. to keep only the cross-validation reports), if you now call :meth:`~skore.Summary.compare`, you get only the :class:`~skore.CrossValidationReport` objects, which you can directly put in the form of a :class:`~skore.ComparisonReport`: .. GENERATED FROM PYTHON SOURCE LINES 436-441 .. code-block:: Python new_report = summary.query('report_type == "cross-validation"').compare( return_as="report" ) new_report .. raw:: html
Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(datetime=DatetimeEncoder(periodic_encoding='spline'))),
                    ('simpleimputer', SimpleImputer(add_indicator=True)),
                    ('squashingscaler', SquashingScaler(max_absolute_value=5)),
                    ('logisticregression', LogisticRegression())])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").

Pipeline(steps=[('tablevectorizer',
                     TableVectorizer(low_cardinality=OrdinalEncoder(handle_unknown='use_encoded_value',
                                                                    unknown_value=-1))),
                    ('randomforestclassifier',
                     RandomForestClassifier(random_state=0))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Please enable javascript

The skrub table reports need javascript to display correctly. If you are displaying a report in a Jupyter notebook and you see this message, you may need to re-execute the cell or to trust the notebook (button on the top right or "File > Trust notebook").



.. GENERATED FROM PYTHON SOURCE LINES 442-456 .. admonition:: Stay tuned! This is only the beginning for skore. We welcome your feedback and ideas to make it the best tool for end-to-end data science. Key benefits of using skore in your ML workflow: * Standardized evaluation and comparison of models * Rich visualizations and diagnostics * Organized experiment tracking * Seamless integration with scikit-learn Feel free to join our community on `Discord `_ or `create an issue `_. .. rst-class:: sphx-glr-timing **Total running time of the script:** (4 minutes 23.643 seconds) .. _sphx_glr_download_auto_examples_getting_started_plot_getting_started.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_getting_started.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_getting_started.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_getting_started.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_