Fetch the repository succeeded.
Ignite is a high-level library to help with training neural networks in PyTorch:
Below we show a side-by-side comparison of using pure pytorch and using ignite to create a training loop to train and validate your model with occasional checkpointing:
As you can see, the code is more concise and readable with ignite. Furthermore, adding additional metrics, or things like early stopping is a breeze in ignite, but can start to rapidly increase the complexity of your code when "rolling your own" training loop.
From pip:
pip install pytorch-ignite
From conda:
conda install ignite -c pytorch
From source:
pip install git+https://github.com/pytorch/ignite
From pip:
pip install --pre pytorch-ignite
From conda (this suggests to install pytorch nightly release instead of stable version as dependency):
conda install ignite -c pytorch-nightly
Ignite's high level of abstraction assumes less about the type of network (or networks) that you are training, and we require the user to define the closure to be run in the training and validation loop. This level of abstraction allows for a great deal more of flexibility, such as co-training multiple models (i.e. GANs) and computing/tracking multiple losses and metrics in your training loop.
The cool thing with handlers is that they offer unparalleled flexibility (compared to say, callbacks). Handlers can be
any function: e.g. lambda, simple function, class method etc. The first argument can be optionally engine, but not necessary.
Thus, we do not require to inherit from an interface and override its abstract methods which could unnecessarily bulk
up your code and its complexity.
trainer.add_event_handler(Events.STARTED, lambda _: print("Start training"))
# attach handler with args, kwargs
mydata = [1, 2, 3, 4]
logger = ...
def on_training_ended(data):
print("Training is ended. mydata={}".format(data))
# User can use variables from another scope
logger.info("Training is ended")
trainer.add_event_handler(Events.COMPLETED, on_training_ended, mydata)
# call any number of functions on a single event
trainer.add_event_handler(Events.COMPLETED, lambda engine: print("OK"))
@trainer.on(Events.ITERATION_COMPLETED)
def log_something(engine):
print(engine.state.output)
# run the validation every 5 epochs
@trainer.on(Events.EPOCH_COMPLETED(every=5))
def run_validation():
# run validation
# change some training variable once on 20th epoch
@trainer.on(Events.EPOCH_STARTED(once=20))
def change_training_variable():
# ...
# Trigger handler with customly defined frequency
@trainer.on(Events.ITERATION_COMPLETED(event_filter=first_x_iters))
def log_gradients():
# ...
Events can be stacked together to enable multiple calls:
@trainer.on(Events.COMPLETED | Events.EPOCH_COMPLETED(every=10))
def run_validation():
# ...
Custom events related to backward and optimizer step calls:
class BackpropEvents(Enum):
BACKWARD_STARTED = 'backward_started'
BACKWARD_COMPLETED = 'backward_completed'
OPTIM_STEP_COMPLETED = 'optim_step_completed'
def update(engine, batch):
# ...
loss = criterion(y_pred, y)
engine.fire_event(BackpropEvents.BACKWARD_STARTED)
loss.backward()
engine.fire_event(BackpropEvents.BACKWARD_COMPLETED)
optimizer.step()
engine.fire_event(BackpropEvents.OPTIM_STEP_COMPLETED)
# ...
trainer = Engine(update)
trainer.register_events(*BackpropEvents)
@trainer.on(BackpropEvents.BACKWARD_STARTED)
def function_before_backprop(engine):
# ...
Metrics for various tasks: Precision, Recall, Accuracy, Confusion Matrix, IoU etc, ~20 regression metrics.
Users can also compose their own metrics with ease from existing ones using arithmetic operations or torch methods:
precision = Precision(average=False)
recall = Recall(average=False)
F1_per_class = (precision * recall * 2 / (precision + recall))
F1_mean = F1_per_class.mean() # torch mean method
F1_mean.attach(engine, "F1")
The code in ignite.contrib is not as fully maintained as the core part of the library.
We provide several examples ported from
pytorch/examples using ignite to display how it helps to write compact and
full-featured training loops in a few lines of code:
Basic neural network training on MNIST dataset with/without ignite.contrib module:
Text Classification using Convolutional Neural
Networks Variational Auto
Encoders Convolutional Neural Networks for Classifying Fashion-MNIST
Dataset Training Cycle-GAN on Horses to
Zebras Finetuning EfficientNet-B0 on
CIFAR100 Hyperparameters tuning with
Ax Basic example of LR finder on
MNIST Benchmark mixed precision training on Cifar100:
torch.cuda.amp vs nvidia/apex MNIST training on a single
TPUTraining a small variant of ResNet on CIFAR10 in various configurations: 1) single gpu, 2) single node multiple gpus, 3) multiple nodes and multilple gpus.
Inspired by torchvision/references, we provide several reproducible baselines for vision tasks:
Features:
GitHub issues: questions, bug reports, feature requests, etc.
Discuss.PyTorch, category "Ignite".
PyTorch Slack at #pytorch-ignite channel. Request access.
We have created a form for "user feedback". We appreciate any type of feedback and this is how we would like to see our community:
Thank you !
Please see the contribution guidelines for more information.
As always, PRs are welcome
See other projects at "Used by"
If your project implements a paper, represents other use-cases not covered in our official tutorials, Kaggle competition's code or just your code presents interesting results and uses Ignite. We would like to add your project in this list, so please send a PR with brief description of the project.
Project is currently maintained by a team of volunteers. See the "About us" page for a list of core contributors.
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手