代码拉取完成，页面将自动刷新
Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grokked codebase in which users can freely experiment with wild ideas (speculative research).
Our design principles are:
In the spirit of these principles, this first version focuses on supporting the state-of-the-art, single-GPU Rainbow agent (Hessel et al., 2018) applied to Atari 2600 game-playing (Bellemare et al., 2013). Specifically, our Rainbow agent implements the three components identified as most important by Hessel et al.:
For completeness, we also provide an implementation of DQN (Mnih et al., 2015). For additional details, please see our documentation.
We provide a set of Colaboratory notebooks which demonstrate how to use Dopamine.
We provide a website which displays the learning curves for all the provided agents, on all the games.
This is not an official Google product.
16/102020: Learning curves for the QR-DQN JAX agent have been added to the baseline plots!
03/08/2020: Dopamine now supports JAX agents! This includes an implementation of the Quantile Regression agent (QR-DQN) which has been a common request. Find out more in our jax subdirectory, which includes trained agent checkpoints.
27/07/2020: Dopamine now runs on TensorFlow 2. However, Dopamine is still written as TensorFlow 1.X code. This means your project may need to explicity disable TensorFlow 2 behaviours with:
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
if you are using custom entry-point for training your agent. The migration to TensorFlow 2 also means that Dopamine no longer supports Python 2.
02/09/2019: Dopamine has switched its network definitions to use
tf.keras.Model. The previous tf.contrib.slim based networks are removed.
If your agents inherit from dopamine agents you need to update your code.
._get_network_type() and ._network_template() functions are replaced
with ._create_network() and network_type definitions are moved inside
the model definition.
# The following two functions are replaced with `_create_network()`.
# def _get_network_type(self):
# return collections.namedtuple('DQN_network', ['q_values'])
# def _network_template(self, state):
# return self.network(self.num_actions, self._get_network_type(), state)
def _create_network(self, name):
"""Builds the convolutional network used to compute the agent's Q-values.
Args:
name: str, this name is passed to the tf.keras.Model and used to create
variable scope under the hood by the tf.keras.Model.
Returns:
network: tf.keras.Model, the network instantiated by the Keras model.
"""
# `self.network` is set to `atari_lib.NatureDQNNetwork`.
network = self.network(self.num_actions, name=name)
return network
def _build_networks(self):
# The following two lines are replaced.
# self.online_convnet = tf.make_template('Online', self._network_template)
# self.target_convnet = tf.make_template('Target', self._network_template)
self.online_convnet = self._create_network(name='Online')
self.target_convnet = self._create_network(name='Target')
If your code overwrites ._network_template(), ._get_network_type() or
._build_networks() make sure you update your code to fit with the new
API. If your code overwrites ._build_networks() you need to replace
tf.make_template('Online', self._network_template) with
self._create_network(name='Online').
The variables of each network can be obtained from the networks as follows:
vars = self.online_convnet.variables.
Baselines and older checkpoints can be loaded by adding the following line to your gin file.
atari_lib.maybe_transform_variable_names.legacy_checkpoint_load = True
11/06/2019: Visualization utilities added to generate videos and still images of a trained agent interacting with its environment. See an example colaboratory here.
30/01/2019: Dopamine 2.0 now supports general discrete-domain gym environments.
01/11/2018: Download links for each individual checkpoint, to avoid having to download all of the checkpoints.
29/10/2018: Graph definitions now show up in Tensorboard.
16/10/2018: Fixed a subtle bug in the IQN implementation and upated the colab tools, the JSON files, and all the downloadable data.
18/09/2018: Added support for double-DQN style updates for the
ImplicitQuantileAgent.
double_dqn constructor parameter.18/09/2018: Added support for reporting in-iteration losses directly from the agent to Tensorboard.
run_experiment.create_agent.debug_mode = True via the
configuration file or using the gin_bindings flag to enable it.summary_writing_frequency
agent constructor parameter (defaults to 500).27/08/2018: Dopamine launched!
Installing from source allows you to modify the agents and experiments as you please, and is likely to be the pathway of choice for long-term use. The instructions below assume that you will be running Dopamine in a virtual environment. A virtual environment lets you control which dependencies are installed for which program.
Dopamine is a Tensorflow-based framework, and we recommend you also consult the Tensorflow documentation for additional details. Finally, these instructions are for Python 3.6 and above.
First download the Dopamine source.
git clone https://github.com/google/dopamine.git
Then create a virtual environment and activate it.
python3 -m venv ./dopamine-venv
source dopamine-venv/bin/activate
Finally setup the environment and install Dopamine's dependencies
pip install -U pip
pip install -r dopamine/requirements.txt
You can test whether the installation was successful by running the following:
cd dopamine
export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_test
The entry point to the standard Atari 2600 experiment is
dopamine/discrete_domains/train.py.
To run the basic DQN agent,
python -um dopamine.discrete_domains.train \
--base_dir /tmp/dopamine_runs \
--gin_files dopamine/agents/dqn/configs/dqn.gin
By default, this will kick off an experiment lasting 200 million frames. The command-line interface will output statistics about the latest training episode:
[...]
I0824 17:13:33.078342 140196395337472 tf_logging.py:115] gamma: 0.990000
I0824 17:13:33.795608 140196395337472 tf_logging.py:115] Beginning training...
Steps executed: 5903 Episode length: 1203 Return: -19.
To get finer-grained information about the process,
you can adjust the experiment parameters in
dopamine/agents/dqn/configs/dqn.gin,
in particular by reducing Runner.training_steps and Runner.evaluation_steps,
which together determine the total number of steps needed to complete an
iteration. This is useful if you want to inspect log files or checkpoints, which
are generated at the end of each iteration.
More generally, the whole of Dopamine is easily configured using the gin configuration framework.
We provide sample configuration files for training an agent on Cartpole and Acrobot. For example, to train C51 on Cartpole with default settings, run the following command:
python -um dopamine.discrete_domains.train \
--base_dir /tmp/dopamine_runs \
--gin_files dopamine/agents/rainbow/configs/c51_cartpole.gin
You can train Rainbow on Acrobot with the following command:
python -um dopamine.discrete_domains.train \
--base_dir /tmp/dopamine_runs \
--gin_files dopamine/agents/rainbow/configs/rainbow_acrobot.gin
An easy, alternative way to install Dopamine is as a Python library:
pip install dopamine-rl
Mnih et al., Human-level Control through Deep Reinforcement Learning. Nature, 2015.
If you use Dopamine in your work, we ask that you cite our white paper. Here is an example BibTeX entry:
@article{castro18dopamine,
author = {Pablo Samuel Castro and
Subhodeep Moitra and
Carles Gelada and
Saurabh Kumar and
Marc G. Bellemare},
title = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},
year = {2018},
url = {http://arxiv.org/abs/1812.06110},
archivePrefix = {arXiv}
}
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。