Metadata-Version: 2.1
Name: optuna
Version: 3.5.0
Summary: A hyperparameter optimization framework
Author: Takuya Akiba
License: MIT License
        
        Copyright (c) 2018 Preferred Networks, Inc.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
        
        ==
        
        Optuna contains code that is licensed by third-party developers.
        
        ==
        SciPy
        
        
        The Optuna contains the codes from SciPy project.
        
        
        Copyright (c) 2001-2002 Enthought, Inc. 2003-2022, SciPy Developers.
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions
        are met:
        
        1. Redistributions of source code must retain the above copyright
           notice, this list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above
           copyright notice, this list of conditions and the following
           disclaimer in the documentation and/or other materials provided
           with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived
           from this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
        "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
        LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
        A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
        OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
        SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
        LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
        DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
        THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
        (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
        
        
        ==
        
        fdlibm
        
         Copyright (C) 1993 by Sun Microsystems, Inc. All rights reserved.
        
         Developed at SunPro, a Sun Microsystems, Inc. business.
         Permission to use, copy, modify, and distribute this
         software is freely granted, provided that this notice
         is preserved.
        
        
Project-URL: homepage, https://optuna.org/
Project-URL: repository, https://github.com/optuna/optuna
Project-URL: documentation, https://optuna.readthedocs.io
Project-URL: bugtracker, https://github.com/optuna/optuna/issues
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML
Requires-Dist: alembic >=1.5.0
Requires-Dist: colorlog
Requires-Dist: numpy
Requires-Dist: packaging >=20.0
Requires-Dist: sqlalchemy >=1.3.0
Requires-Dist: tqdm
Provides-Extra: benchmark
Requires-Dist: asv >=0.5.0 ; extra == 'benchmark'
Requires-Dist: botorch ; extra == 'benchmark'
Requires-Dist: cma ; extra == 'benchmark'
Requires-Dist: scikit-optimize ; extra == 'benchmark'
Requires-Dist: virtualenv ; extra == 'benchmark'
Provides-Extra: checking
Requires-Dist: black ; extra == 'checking'
Requires-Dist: blackdoc ; extra == 'checking'
Requires-Dist: flake8 ; extra == 'checking'
Requires-Dist: isort ; extra == 'checking'
Requires-Dist: mypy ; extra == 'checking'
Requires-Dist: mypy-boto3-s3 ; extra == 'checking'
Requires-Dist: types-PyYAML ; extra == 'checking'
Requires-Dist: types-redis ; extra == 'checking'
Requires-Dist: types-setuptools ; extra == 'checking'
Requires-Dist: types-tqdm ; extra == 'checking'
Requires-Dist: typing-extensions >=3.10.0.0 ; extra == 'checking'
Provides-Extra: document
Requires-Dist: ase ; extra == 'document'
Requires-Dist: botorch ; extra == 'document'
Requires-Dist: cma ; extra == 'document'
Requires-Dist: cmaes >=0.10.0 ; extra == 'document'
Requires-Dist: distributed ; extra == 'document'
Requires-Dist: fvcore ; extra == 'document'
Requires-Dist: lightgbm ; extra == 'document'
Requires-Dist: matplotlib !=3.6.0 ; extra == 'document'
Requires-Dist: mlflow ; extra == 'document'
Requires-Dist: pandas ; extra == 'document'
Requires-Dist: pillow ; extra == 'document'
Requires-Dist: plotly >=4.9.0 ; extra == 'document'
Requires-Dist: scikit-learn ; extra == 'document'
Requires-Dist: scikit-optimize ; extra == 'document'
Requires-Dist: sphinx ; extra == 'document'
Requires-Dist: sphinx-copybutton ; extra == 'document'
Requires-Dist: sphinx-gallery ; extra == 'document'
Requires-Dist: sphinx-plotly-directive ; extra == 'document'
Requires-Dist: sphinx-rtd-theme >=1.2.0 ; extra == 'document'
Requires-Dist: torch ; extra == 'document'
Requires-Dist: torchaudio ; extra == 'document'
Requires-Dist: torchvision ; extra == 'document'
Provides-Extra: integration
Requires-Dist: botorch >=0.4.0 ; extra == 'integration'
Requires-Dist: cma ; extra == 'integration'
Requires-Dist: distributed ; extra == 'integration'
Requires-Dist: lightgbm ; extra == 'integration'
Requires-Dist: lightning ; extra == 'integration'
Requires-Dist: mlflow ; extra == 'integration'
Requires-Dist: pandas ; extra == 'integration'
Requires-Dist: pytorch-ignite ; extra == 'integration'
Requires-Dist: scikit-learn >=0.24.2 ; extra == 'integration'
Requires-Dist: scikit-optimize ; extra == 'integration'
Requires-Dist: shap ; extra == 'integration'
Requires-Dist: tensorflow ; extra == 'integration'
Requires-Dist: torch ; extra == 'integration'
Requires-Dist: torchaudio ; extra == 'integration'
Requires-Dist: torchvision ; extra == 'integration'
Requires-Dist: wandb ; extra == 'integration'
Requires-Dist: xgboost ; extra == 'integration'
Requires-Dist: catboost >=0.26 ; (sys_platform != "darwin") and extra == 'integration'
Requires-Dist: catboost <1.2,>=0.26 ; (sys_platform == "darwin") and extra == 'integration'
Provides-Extra: optional
Requires-Dist: boto3 ; extra == 'optional'
Requires-Dist: cmaes >=0.10.0 ; extra == 'optional'
Requires-Dist: google-cloud-storage ; extra == 'optional'
Requires-Dist: matplotlib !=3.6.0 ; extra == 'optional'
Requires-Dist: pandas ; extra == 'optional'
Requires-Dist: plotly >=4.9.0 ; extra == 'optional'
Requires-Dist: redis ; extra == 'optional'
Requires-Dist: scikit-learn >=0.24.2 ; extra == 'optional'
Requires-Dist: botorch ; (python_version <= "3.11") and extra == 'optional'
Provides-Extra: test
Requires-Dist: coverage ; extra == 'test'
Requires-Dist: fakeredis[lua] ; extra == 'test'
Requires-Dist: kaleido ; extra == 'test'
Requires-Dist: moto ; extra == 'test'
Requires-Dist: pytest ; extra == 'test'
Requires-Dist: scipy >=1.9.2 ; (python_version >= "3.8") and extra == 'test'

<div align="center"><img src="https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png" width="800"/></div>

# Optuna: A hyperparameter optimization framework

[![Python](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org)
[![pypi](https://img.shields.io/pypi/v/optuna.svg)](https://pypi.python.org/pypi/optuna)
[![conda](https://img.shields.io/conda/vn/conda-forge/optuna.svg)](https://anaconda.org/conda-forge/optuna)
[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/optuna/optuna)
[![Read the Docs](https://readthedocs.org/projects/optuna/badge/?version=stable)](https://optuna.readthedocs.io/en/stable/)
[![Codecov](https://codecov.io/gh/optuna/optuna/branch/master/graph/badge.svg)](https://codecov.io/gh/optuna/optuna)

[**Website**](https://optuna.org/)
| [**Docs**](https://optuna.readthedocs.io/en/stable/)
| [**Install Guide**](https://optuna.readthedocs.io/en/stable/installation.html)
| [**Tutorial**](https://optuna.readthedocs.io/en/stable/tutorial/index.html)
| [**Examples**](https://github.com/optuna/optuna-examples)

*Optuna* is an automatic hyperparameter optimization software framework, particularly designed
for machine learning. It features an imperative, *define-by-run* style user API. Thanks to our
*define-by-run* API, the code written with Optuna enjoys high modularity, and the user of
Optuna can dynamically construct the search spaces for the hyperparameters.

## Key Features

Optuna has modern functionalities as follows:

- [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/001_first.html)
  - Handle a wide variety of tasks with a simple installation that has few requirements.
- [Pythonic search spaces](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/002_configurations.html)
  - Define search spaces using familiar Python syntax including conditionals and loops.
- [Efficient optimization algorithms](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/003_efficient_optimization_algorithms.html)
  - Adopt state-of-the-art algorithms for sampling hyperparameters and efficiently pruning unpromising trials.
- [Easy parallelization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html)
  - Scale studies to tens or hundreds or workers with little or no changes to the code.
- [Quick visualization](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/005_visualization.html)
  - Inspect optimization histories from a variety of plotting functions.


## Basic Concepts

We use the terms *study* and *trial* as follows:

- Study: optimization based on an objective function
- Trial: a single execution of the objective function

Please refer to sample code below. The goal of a *study* is to find out the optimal set of
hyperparameter values (e.g., `regressor` and `svr_c`) through multiple *trials* (e.g.,
`n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the
optimization *studies*.

[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna-examples/blob/main/quickstart.ipynb)

```python
import ...

# Define an objective function to be minimized.
def objective(trial):

    # Invoke suggest methods of a Trial object to generate hyperparameters.
    regressor_name = trial.suggest_categorical('regressor', ['SVR', 'RandomForest'])
    if regressor_name == 'SVR':
        svr_c = trial.suggest_float('svr_c', 1e-10, 1e10, log=True)
        regressor_obj = sklearn.svm.SVR(C=svr_c)
    else:
        rf_max_depth = trial.suggest_int('rf_max_depth', 2, 32)
        regressor_obj = sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)

    X, y = sklearn.datasets.fetch_california_housing(return_X_y=True)
    X_train, X_val, y_train, y_val = sklearn.model_selection.train_test_split(X, y, random_state=0)

    regressor_obj.fit(X_train, y_train)
    y_pred = regressor_obj.predict(X_val)

    error = sklearn.metrics.mean_squared_error(y_val, y_pred)

    return error  # An objective value linked with the Trial object.

study = optuna.create_study()  # Create a new study.
study.optimize(objective, n_trials=100)  # Invoke optimization of the objective function.
```


## Installation

Optuna is available at [the Python Package Index](https://pypi.org/project/optuna/) and on [Anaconda Cloud](https://anaconda.org/conda-forge/optuna).

```bash
# PyPI
$ pip install optuna
```

```bash
# Anaconda Cloud
$ conda install -c conda-forge optuna
```

Optuna supports Python 3.7 or newer.

Also, we also provide Optuna docker images on [DockerHub](https://hub.docker.com/r/optuna/optuna).


## Examples

Examples can be found in [optuna/optuna-examples](https://github.com/optuna/optuna-examples).

## Integrations

Optuna has integration features with various third-party libraries. Integrations can be found in [optuna/optuna-integration](https://github.com/optuna/optuna-integration) and the document is available [here](https://optuna-integration.readthedocs.io/en/stable/index.html). Integrations support libraries such as the following:

* [Catalyst](https://github.com/optuna/optuna-examples/tree/main/pytorch/catalyst_simple.py)
* [Catboost](https://github.com/optuna/optuna-examples/tree/main/catboost/catboost_pruning.py)
* [Dask](https://github.com/optuna/optuna-examples/tree/main/dask/dask_simple.py)
* [fastai (v2)](https://github.com/optuna/optuna-examples/tree/main/fastai/fastaiv2_simple.py)
* [Keras](https://github.com/optuna/optuna-examples/tree/main/keras/keras_integration.py)
* [LightGBM](https://github.com/optuna/optuna-examples/tree/main/lightgbm/lightgbm_integration.py)
* [MLflow](https://github.com/optuna/optuna-examples/tree/main/mlflow/keras_mlflow.py)
* [MXNet](https://github.com/optuna/optuna-examples/tree/main/mxnet/mxnet_integration.py)
* [PyTorch](https://github.com/optuna/optuna-examples/tree/main/pytorch/pytorch_simple.py)
* [PyTorch Ignite](https://github.com/optuna/optuna-examples/tree/main/pytorch/pytorch_ignite_simple.py)
* [PyTorch Lightning](https://github.com/optuna/optuna-examples/tree/main/pytorch/pytorch_lightning_simple.py)
* [TensorBoard](https://github.com/optuna/optuna-examples/tree/main/tensorboard/tensorboard_simple.py)
* [TensorFlow](https://github.com/optuna/optuna-examples/tree/main/tensorflow/tensorflow_estimator_integration.py)
* [tf.keras](https://github.com/optuna/optuna-examples/tree/main/tfkeras/tfkeras_integration.py)
* [Weights & Biases](https://github.com/optuna/optuna-examples/tree/main/wandb/wandb_integration.py)
* [XGBoost](https://github.com/optuna/optuna-examples/tree/main/xgboost/xgboost_integration.py)


## Web Dashboard

[Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a real-time web dashboard for Optuna.
You can check the optimization history, hyperparameter importances, etc. in graphs and tables.
You don't need to create a Python script to call [Optuna's visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html) functions.
Feature requests and bug reports welcome!

![optuna-dashboard](https://user-images.githubusercontent.com/5564044/204975098-95c2cb8c-0fb5-4388-abc4-da32f56cb4e5.gif)

Install `optuna-dashboard` via pip:

```
$ pip install optuna-dashboard
$ optuna-dashboard sqlite:///db.sqlite3
...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
```


## Communication

- [GitHub Discussions] for questions.
- [GitHub Issues] for bug reports and feature requests.

[GitHub Discussions]: https://github.com/optuna/optuna/discussions
[GitHub issues]: https://github.com/optuna/optuna/issues


## Contribution

Any contributions to Optuna are more than welcome!

If you are new to Optuna, please check the [good first issues](https://github.com/optuna/optuna/labels/good%20first%20issue). They are relatively simple, well-defined and are often good starting points for you to get familiar with the contribution workflow and other developers.

If you already have contributed to Optuna, we recommend the other [contribution-welcome issues](https://github.com/optuna/optuna/labels/contribution-welcome).

For general guidelines how to contribute to the project, take a look at [CONTRIBUTING.md](./CONTRIBUTING.md).


## Reference

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019.
Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902)).
