tutorials/02. Built-in Data Iterators.zepp.ipynb · wangyingdong/deeplearning4j-examples

Create your Gitee Account

Explore and code with more than 13.5 million developers，Free private repositories ！：）

文件

.github

android

contrib

datavec-examples

dl4j-cuda-specific-examples

dl4j-examples

dl4j-examples_javafx

dl4j-spark-examples

jumpy-examples

lstm-hdfs

nd4j-examples

pydatavec-examples

rl4j-examples

shared-utilities

standalone-sample-project

tf-import-examples

tutorials

docker

00. Quickstart for Deeplearning4j.json

00. Quickstart for Deeplearning4j.zepp.ipynb

01. MultiLayerNetwork and ComputationGraph.json

01. MultiLayerNetwork and ComputationGraph.zepp.ipynb

02. Built-in Data Iterators.json

02. Built-in Data Iterators.zepp.ipynb

03. Logistic Regression.json

03. Logistic Regression.zepp.ipynb

04. Feed-forward.json

04. Feed-forward.zepp.ipynb

05. Basic Autoencoder- Anomaly Detection Using Reconstruction Error.json

05. Basic Autoencoder- Anomaly Detection Using Reconstruction Error.zepp.ipynb

06. Advanced Autoencoder- Trajectory Clustering using AIS.json

06. Advanced Autoencoder- Trajectory Clustering using AIS.zepp.ipynb

07. Convolutions- Train FaceNet Using Center Loss.json

07. Convolutions- Train FaceNet Using Center Loss.zepp.ipynb

08. RNNs- Sequence Classification of Synthetic Control Data.json

08. RNNs- Sequence Classification of Synthetic Control Data.zepp.ipynb

09. Early Stopping.ipynb

09. Early Stopping.zepp.json

10. Layers and Preprocessors.ipynb

10. Layers and Preprocessors.zepp.json

11. Hyperparameter Optimization.json

11. Hyperparameter Optimization.zepp.ipynb

12. Clinical Time Series LSTM Example.json

12. Clinical Time Series LSTM Example.zepp.ipynb

13. Clinical LSTM Time Series Example Using SKIL.json

13. Clinical LSTM Time Series Example Using SKIL.zepp.ipynb

14. Parallel Training.json

14. Parallel Training.zepp.ipynb

15. Sea Temperature Convolutional LSTM Example.json

15. Sea Temperature Convolutional LSTM Example.zepp.ipynb

16. Sea Temperature Convolutional LSTM Example 2.json

16. Sea Temperature Convolutional LSTM Example 2.zepp.ipynb

17. Instacart Multitask Example.json

17. Instacart Multitask Example.zepp.ipynb

18. Instacart Single Task Example.json

18. Instacart Single Task Example.zepp.ipynb

19. Cloud Detection Example.json

19. Cloud Detection Example.zepp.ipynb

Dockerfile

README.md

.editorconfig

.gitignore

CONTRIBUTORS.md

LICENSE.txt

README.md

pom.xml

runexamples.sh

Clone or Download

02. Built-in Data Iterators.zepp.ipynb 9.72 KB

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note\n",
    "\n",
    "Please view the [README](https://github.com/eclipse/deeplearning4j-examples/blob/master/tutorials/README.md) to learn about installing, setting up dependencies, and importing notebooks in Zeppelin."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Background\n",
    "\n",
    "Toy datasets are essential for testing hypotheses and getting started with any neural network training process. Deeplearning4j comes with built-in dataset iterators for common datasets, including but not limited to:\n",
    "\n",
    "- MNIST\n",
    "- Iris\n",
    "- TinyImageNet (subset of ImageNet)\n",
    "- CIFAR-10\n",
    "- Labelled Faces in the Wild\n",
    "- Curve Fragment Ground-Truth Dataset\n",
    "\n",
    "These datasets are also used as a baseline for testing other machine learning algorithms. Please remember to use these datasets correctly within the terms of their license (for example, you must obtain special permission to use ImageNet in a commercial project)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What are we going to learn in this tutorial?\n",
    "\n",
    "Building on what we know about `MultiLayerNetwork` and `ComputationGraph`, we will instantiate a couple data iterators to feed a toy dataset into a neural network for training. This tutorial is focused on training a classifier (you can also train networks for regression, or use them for unsupervised training via an autoencoder), and you will also learn how to interpret the output in the console."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "autoscroll": "auto"
   },
   "source": [
    "import org.nd4j.linalg.activations.Activation\n",
    "import org.nd4j.linalg.dataset.api.iterator.DataSetIterator\n",
    "import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator\n",
    "import org.deeplearning4j.eval.Evaluation\n",
    "import org.deeplearning4j.nn.api.OptimizationAlgorithm\n",
    "import org.deeplearning4j.nn.conf.MultiLayerConfiguration\n",
    "import org.deeplearning4j.nn.conf.NeuralNetConfiguration\n",
    "import org.deeplearning4j.nn.conf.Updater\n",
    "import org.deeplearning4j.nn.conf.layers.DenseLayer\n",
    "import org.deeplearning4j.nn.conf.layers.OutputLayer\n",
    "import org.deeplearning4j.nn.multilayer.MultiLayerNetwork\n",
    "import org.deeplearning4j.nn.weights.WeightInit\n",
    "import org.deeplearning4j.optimize.listeners.ScoreIterationListener\n",
    "import org.nd4j.linalg.api.ndarray.INDArray\n",
    "import org.nd4j.linalg.dataset.DataSet\n",
    "import org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction\n",
    "import org.slf4j.Logger\n",
    "import org.slf4j.LoggerFactory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The MNIST classifier network\n",
    "\n",
    "A `MultiLayerNetwork` can classify MNIST digits. If you are not familiar with MNIST, it is a dataset originally assembled for recognizing hand-written numerals. You can read more about MNIST [here](https://en.wikipedia.org/wiki/MNIST_database).\n",
    "\n",
    "Once you have imported what you need, set up a basic MultiLayerNetwork like below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "autoscroll": "auto"
   },
   "outputs": [],
   "source": [
    "//number of rows and columns in the input pictures\n",
    "val numRows = 28\n",
    "val numColumns = 28\n",
    "val outputNum = 10 // number of output classes\n",
    "val batchSize = 128 // batch size for each epoch\n",
    "val rngSeed = 123 // random number seed for reproducibility\n",
    "val numEpochs = 15 // number of epochs to perform\n",
    "\n",
    "val conf: MultiLayerConfiguration = new NeuralNetConfiguration.Builder()\n",
    "    .seed(rngSeed) //include a random seed for reproducibility\n",
    "    .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT) // use stochastic gradient descent as an optimization algorithm\n",
    "    .iterations(1) // in nearly all cases should be 1\n",
    "    .learningRate(0.006) //specify the learning rate\n",
    "    .updater(Updater.NESTEROVS).momentum(0.9) //specify the rate of change of the learning rate.\n",
    "    .regularization(true).l2(1e-4)\n",
    "    .list()\n",
    "    .layer(0, new DenseLayer.Builder() //create the first, input layer with xavier initialization\n",
    "            .nIn(numRows * numColumns)\n",
    "            .nOut(1000)\n",
    "            .activation(Activation.RELU)\n",
    "            .weightInit(WeightInit.XAVIER)\n",
    "            .build())\n",
    "    .layer(1, new OutputLayer.Builder(LossFunction.NEGATIVELOGLIKELIHOOD) //create hidden layer\n",
    "            .nIn(1000)\n",
    "            .nOut(outputNum)\n",
    "            .activation(Activation.SOFTMAX)\n",
    "            .weightInit(WeightInit.XAVIER)\n",
    "            .build())\n",
    "    .build()\n",
    "\n",
    "val model = new MultiLayerNetwork(conf)\n",
    "model.init()\n",
    "//print the score with every 1 iteration\n",
    "model.setListeners(new ScoreIterationListener(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the MNIST iterator\n",
    "\n",
    "The MNIST iterator, like most of Deeplearning4j's built-in iterators, extends the `DataSetIterator` class. This API allows for simple instantiation of datasets and automatic downloading of data in the background. The MNIST data iterator API specifically allows you to specify whether you are using the training or testing dataset, so instantiate two different iterators to evaluate your network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "autoscroll": "auto"
   },
   "outputs": [],
   "source": [
    "//Get the DataSetIterators:\n",
    "val mnistTrain = new MnistDataSetIterator(batchSize, true, rngSeed)\n",
    "val mnistTest = new MnistDataSetIterator(batchSize, false, rngSeed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " \n",
    "\n",
    "### Performing basic training\n",
    "\n",
    "Now that the network configuration is set up and instantiated along with our MNIST test/train iterators, training takes just a few lines of code. The fun begins.\n",
    "\n",
    "Earlier we attached a `ScoreIterationListener` to the model by using the `setListeners()` method. Depending on the browser you are using to run this notebook, you can open the debugger/inspector to view listener output. This output is redirected to the console since the internals of Deeplearning4j use SL4J for logging, and the output is being redirected by Zeppelin. This is a good thing since it can reduce clutter in notebooks.\n",
    "\n",
    "As a well-tuned model continues to train, its error score will decrease with each iteration. This error or loss score will eventually converge to a value close to zero. Note that more complex networks and problems may never yield an optimal score. This is where you need to become the expert and continue to tune and change your model's configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "autoscroll": "auto"
   },
   "outputs": [],
   "source": [
    "// the \"simple\" way to do multiple epochs is to wrap fit() in a loop\n",
    "(1 to numEpochs).foreach(_ => model.fit(mnistTrain) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating the model\n",
    "\n",
    "\"Overfitting\" is a common problem in deep learning where your model doesn't generalize well to the problem you are trying to solve. This can happen when you have run the algorithm for too many epochs over a training dataset, when you haven't used a regularization technique like [Dropout](https://en.wikipedia.org/wiki/Dropout_%28neural_networks%29), or the training dataset isn't big enough and doesn't encapsulate all of the features that are descriptive of your classes in the real world.\n",
    "\n",
    "Deeplearning4j comes with built-in tools for model evaluation. The simplest is to pass a testing iterator to `eval()` and retrieve an `Evaluation` object. Many more, including ROC plotting and regression evaluation, are available in the [org.deeplearning4j.eval](https://github.com/eclipse/deeplearning4j/tree/master/deeplearning4j-core/src/test/java/org/deeplearning4j/eval) package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "autoscroll": "auto"
   },
   "outputs": [],
   "source": [
    "val evaluation = model.evaluate(mnistTest)\n",
    "\n",
    "// print the basic statistics about the trained classifier\n",
    "println(\"Accuracy: \"+evaluation.accuracy())\n",
    "println(\"Precision: \"+evaluation.precision())\n",
    "println(\"Recall: \"+evaluation.recall())\n",
    "\n",
    "// in more complex scenarios, a confusion matrix is quite helpful\n",
    "println(evaluation.confusionToString())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What's next?\n",
    "\n",
    "- Check out all of our tutorials available [on Github](https://github.com/eclipse/deeplearning4j-examples/tree/master/tutorials)."
   ]
  }
 ],
 "metadata": {
  "zeppelin": {
   "name":"Built-in Data Iterators",
   "id":"2CY22DZ1M"
  },
  "kernelspec": {
   "display_name": "Spark 2.0.0 - Scala 2.11",
   "language": "scala",
   "name": "spark2-scala"
  },
  "language_info": {
   "codemirror_mode": "text/x-scala",
   "file_extension": ".scala",
   "mimetype": "text/x-scala",
   "name": "scala",
   "pygments_lexer": "scala",
   "version": "2.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

Copy Edit Raw Blame History

authored 6 years ago . Update links

文件格式错误，请检查文件是否为ipynb格式

马建仓 AI 助手

尝试更多

代码解读

代码找茬

代码优化

https://gitee.com/dwwwcn/deeplearning4j-examples.git

git@gitee.com:dwwwcn/deeplearning4j-examples.git

dwwwcn

deeplearning4j-examples

master

wangyingdong/deeplearning4j-examples

About

Releases

Contributors (13)

Activities