{"paragraphs":[{"text":"%md\n### Note\n\nPlease view the [README](https://github.com/eclipse/deeplearning4j/tree/master/dl4j-examples/tutorials/README.md) to learn about installing, setting up dependencies, and importing notebooks in Zeppelin","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Note</h3>\n<p>Please view the <a href=\"https://github.com/eclipse/deeplearning4j/tree/master/dl4j-examples/tutorials/README.md\">README</a> to learn about installing, setting up dependencies, and importing notebooks in Zeppelin</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839190_-1675012213","id":"20180427-083911_1128930448","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:7228"},{"text":"%md\n\n### Background\n\nThis tutorial will be similar to the Instacart Multitask tutorial. The only difference is that we will not use multitasking to train our neural network. Recall the data originially comes from a Kaggle challenge (kaggle.com/c/instacart-market-basket-analysis). We removed users that only made 1 order using the instacart app and then took 5000 users out of the remaining to be part of the data for this tutorial. \n\nFor each order, we have information on the product the user purchased. For example, there is information on the product name, what aisle it is found in, and the department it falls under. To construct features, we extracted indicators representing whether or not a user purchased a product in the given aisles for each order. In total there are 134 aisles. The targets were whether or not a user will buy a product in the breakfast department in the next order. As mentioned, we will not use any auxiliary targets.\n\nBecause of temporal dependencies within the data, we used a LSTM network for our model.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Background</h3>\n<p>This tutorial will be similar to the Instacart Multitask tutorial. The only difference is that we will not use multitasking to train our neural network. Recall the data originially comes from a Kaggle challenge (kaggle.com/c/instacart-market-basket-analysis). We removed users that only made 1 order using the instacart app and then took 5000 users out of the remaining to be part of the data for this tutorial.</p>\n<p>For each order, we have information on the product the user purchased. For example, there is information on the product name, what aisle it is found in, and the department it falls under. To construct features, we extracted indicators representing whether or not a user purchased a product in the given aisles for each order. In total there are 134 aisles. The targets were whether or not a user will buy a product in the breakfast department in the next order. As mentioned, we will not use any auxiliary targets.</p>\n<p>Because of temporal dependencies within the data, we used a LSTM network for our model.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839191_-1675396962","id":"20180427-085703_34966348","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7229"},{"text":"%md\n### Imports\n","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Imports</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839191_-1675396962","id":"20180427-085648_929291696","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7230"},{"text":"import org.deeplearning4j.nn.api.OptimizationAlgorithm;\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration;\nimport org.deeplearning4j.nn.conf.Updater;\nimport org.deeplearning4j.nn.conf.layers.LSTM;\nimport org.deeplearning4j.nn.weights.WeightInit;\nimport org.nd4j.linalg.activations.Activation;\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer;\nimport org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction;\nimport org.deeplearning4j.nn.conf.GradientNormalization;\nimport org.deeplearning4j.eval.ROC;\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader;\nimport org.datavec.api.records.reader.SequenceRecordReader;\nimport org.datavec.api.split.NumberedFileInputSplit;\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator;\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork;\nimport org.nd4j.linalg.api.ndarray.INDArray;\nimport java.io.File;\nimport java.net.URL;\nimport java.io.BufferedInputStream;\nimport java.io.FileInputStream;\nimport java.io.BufferedOutputStream;\nimport java.io.FileOutputStream;\nimport org.apache.commons.io.FilenameUtils;\nimport org.apache.commons.io.FileUtils;\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream;\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry;","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"import org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration\nimport org.deeplearning4j.nn.conf.Updater\nimport org.deeplearning4j.nn.conf.layers.GravesLSTM\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer\nimport org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction\nimport org.deeplearning4j.nn.conf.GradientNormalization\nimport org.deeplearning4j.eval.ROC\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader\nimport org.datavec.api.records.reader.SequenceRecordReader\nimport org.datavec.api.split.NumberedFileInputSplit\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.nd4j.linalg.api.ndarray.INDArray\nimport java.io.File\nimport java.net.URL\nimport java.io.BufferedInputStream\nimport java.io.FileInputStream\nimport java.io.BufferedOutputStream\nimport java.io.FileOutputStream\nimport org.apache.commons.io.FilenameUtils\nimport org.apache.commons.io.FileUtils\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry\n"}]},"apps":[],"jobName":"paragraph_1529915839192_-1677320707","id":"20180427-083930_917505582","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7231"},{"text":"%md \n\n### Download Data","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Download Data</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839193_-1677705456","id":"20180427-083933_1234393367","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7232"},{"text":"%md\nTo download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>To download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839193_-1677705456","id":"20180427-084225_1613961353","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7233"},{"text":"val DATA_URL = \"https://bpstore1.blob.core.windows.net/tutorials/instacart.tar.gz\"\nval DATA_PATH = FilenameUtils.concat(System.getProperty(\"java.io.tmpdir\"), \"dl4j_instacart/\")","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"DATA_URL: String = https://bpstore1.blob.core.windows.net/tutorials/instacart.tar.gz\nDATA_PATH: String = /tmp/dl4j_instacart/\n"}]},"apps":[],"jobName":"paragraph_1529915839194_-1676551209","id":"20180427-084247_2085419306","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7234"},{"text":"val directory = new File(DATA_PATH)\ndirectory.mkdir() \n\nval archizePath = DATA_PATH + \"instacart.tar.gz\"\nval archiveFile = new File(archizePath)\nval extractedPath = DATA_PATH + \"instacart\" \nval extractedFile = new File(extractedPath)\n\nFileUtils.copyURLToFile(new URL(DATA_URL), archiveFile) ","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"directory: java.io.File = /tmp/dl4j_instacart\nres0: Boolean = false\narchizePath: String = /tmp/dl4j_instacart/instacart.tar.gz\narchiveFile: java.io.File = /tmp/dl4j_instacart/instacart.tar.gz\nextractedPath: String = /tmp/dl4j_instacart/instacart\nextractedFile: java.io.File = /tmp/dl4j_instacart/instacart\n"}]},"apps":[],"jobName":"paragraph_1529915839194_-1676551209","id":"20180427-084313_1474080225","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7235"},{"text":"%md\nWe will then extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directories, and copy the files from the tar.gz file.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We will then extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directories, and copy the files from the tar.gz file.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839194_-1676551209","id":"20180427-084416_1688783838","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7236"},{"text":"var fileCount = 0\nvar dirCount = 0\nval BUFFER_SIZE = 4096\nval tais = new TarArchiveInputStream(new GzipCompressorInputStream( new BufferedInputStream( new FileInputStream(archizePath))))\n\nvar entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n\nwhile(entry != null){\n    if (entry.isDirectory()) {\n        new File(DATA_PATH + entry.getName()).mkdirs()\n        dirCount = dirCount + 1\n        fileCount = 0\n    }\n    else {\n        \n        val data = new Array[scala.Byte](4 * BUFFER_SIZE)\n\n        val fos = new FileOutputStream(DATA_PATH + entry.getName());\n        val dest = new BufferedOutputStream(fos, BUFFER_SIZE);\n        var count = tais.read(data, 0, BUFFER_SIZE)\n        \n        while (count != -1) {\n            dest.write(data, 0, count)\n            count = tais.read(data, 0, BUFFER_SIZE)\n        }\n        \n        dest.close()\n        fileCount = fileCount + 1\n    }\n    if(fileCount % 1000 == 0){\n        print(\".\")\n    }\n    \n    entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n}","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"fileCount: Int = 0\ndirCount: Int = 0\nBUFFER_SIZE: Int = 4096\ntais: org.apache.commons.compress.archivers.tar.TarArchiveInputStream = org.apache.commons.compress.archivers.tar.TarArchiveInputStream@49cbc08f\nentry: org.apache.commons.compress.archivers.tar.TarArchiveEntry = org.apache.commons.compress.archivers.tar.TarArchiveEntry@35281714\n..................."}]},"apps":[],"jobName":"paragraph_1529915839195_-1676935958","id":"20180427-084719_19470875","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7237"},{"text":"%md\n### DataSetIterators","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>DataSetIterators</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839195_-1676935958","id":"20180427-084743_1654181620","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7238"},{"text":"%md\nNext we will convert the raw data (csv files) into DataSetIterators, which will be fed into a neural network. Our training data will have 4000 examples which will be represented by a single DataSetIterator, and the testing data will have 1000 examples which will be represented by a separate DataSetIterator.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Next we will convert the raw data (csv files) into DataSetIterators, which will be fed into a neural network. Our training data will have 4000 examples which will be represented by a single DataSetIterator, and the testing data will have 1000 examples which will be represented by a separate DataSetIterator.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839196_-1678859702","id":"20180427-084756_600711638","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7239"},{"text":"val path = FilenameUtils.concat(DATA_PATH, \"instacart/\") // set parent directory\n\nval featureBaseDir = FilenameUtils.concat(path, \"features\") // set feature directory\nval targetsBaseDir = FilenameUtils.concat(path, \"breakfast\") // set label directory","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"path: String = /tmp/dl4j_instacart/instacart/\nfeatureBaseDir: String = /tmp/dl4j_instacart/instacart/features\ntargetsBaseDir: String = /tmp/dl4j_instacart/instacart/breakfast\n"}]},"apps":[],"jobName":"paragraph_1529915839197_-1679244451","id":"20180427-084821_1455072896","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7240"},{"text":"%md\n\nWe first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. Then the SequenceRecordReaderDataSetIterators can be created using the RecordReaders. Since each example has sequences of different lengths, an alignment mode of align end is needed.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. Then the SequenceRecordReaderDataSetIterators can be created using the RecordReaders. Since each example has sequences of different lengths, an alignment mode of align end is needed.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839197_-1679244451","id":"20180427-091115_1295205739","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7241"},{"text":"val trainFeatures = new CSVSequenceRecordReader(1, \",\");\ntrainFeatures.initialize( new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", 1, 4000));\nval trainLabels = new CSVSequenceRecordReader(1, \" \");\ntrainLabels.initialize(new NumberedFileInputSplit(targetsBaseDir + \"/%d.csv\", 1, 4000));\n\nval train = new SequenceRecordReaderDataSetIterator(trainFeatures, trainLabels, 32,\n    2, false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);\n\nval testFeatures = new CSVSequenceRecordReader(1, \",\");\ntestFeatures.initialize( new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", 4001, 5000));\nval testLabels = new CSVSequenceRecordReader(1, \" \");\ntestLabels.initialize(new NumberedFileInputSplit(targetsBaseDir + \"/%d.csv\", 4001, 5000));\n\nval test = new SequenceRecordReaderDataSetIterator(testFeatures, testLabels, 32,\n    2, false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);;","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"trainFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@10e22fed\ntrainLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@40512ac3\ntrain: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@37613d1e\ntestFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@43fbece1\ntestLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@11d0d92e\ntest: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@7c95230a\n"}]},"apps":[],"jobName":"paragraph_1529915839198_-1678090204","id":"20180427-084856_1924041114","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7242"},{"text":"%md \n\n### Neural Network","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Neural Network</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839198_-1678090204","id":"20180427-091302_1119629475","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7243"},{"text":"%md\nThe next task is to set up the neural network configuration. We will use a MultiLayerNetwork and the configuration will be similar to the multitask model from before. Again we use one GravesLSTM layer but this time only one RnnOutputLayer.\n","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>The next task is to set up the neural network configuration. We will use a MultiLayerNetwork and the configuration will be similar to the multitask model from before. Again we use one GravesLSTM layer but this time only one RnnOutputLayer.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839199_-1678474953","id":"20180427-091406_433497358","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7244"},{"text":"val conf = new NeuralNetConfiguration.Builder()\n    .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n    .seed(12345)\n    .dropOut(0.25)\n    .weightInit(WeightInit.XAVIER)\n    .updater(Updater.ADAM)\n    .list()\n    .layer(0, new LSTM.Builder()\n        .activation(Activation.TANH)\n        .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n        .gradientNormalizationThreshold(10)\n        .nIn(134)\n        .nOut(150)\n        .build())\n    .layer(1, new RnnOutputLayer.Builder(LossFunction.XENT)\n        .activation(Activation.SOFTMAX)\n        .nIn(150)\n        .nOut(2)\n        .build())\n.pretrain(false).backprop(true)\n.build();","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"warning: there were 1 deprecation warning(s); re-run with -deprecation for details\nconf: org.deeplearning4j.nn.conf.MultiLayerConfiguration = \n{\n  \"backprop\" : true,\n  \"backpropType\" : \"Standard\",\n  \"cacheMode\" : \"NONE\",\n  \"confs\" : [ {\n    \"cacheMode\" : \"NONE\",\n    \"epochCount\" : 0,\n    \"iterationCount\" : 0,\n    \"l1ByParam\" : { },\n    \"l2ByParam\" : { },\n    \"layer\" : {\n      \"gravesLSTM\" : {\n        \"activationFn\" : {\n          \"TanH\" : { }\n        },\n        \"biasInit\" : 0.0,\n        \"biasUpdater\" : null,\n        \"constraints\" : null,\n        \"dist\" : null,\n        \"forgetGateBiasInit\" : 1.0,\n        \"gateActivationFn\" : {\n          \"Sigmoid\" : { }\n        },\n        \"gradientNormalization\" : \"ClipElementWiseAbsoluteValue\",\n        \"gradientNormalizationThreshold\" : 10.0,\n        \"idropout\" : {\n          \"@class\" : \"org.deeplearning4j.nn.conf.dropout.Dropout\",\n     ..."}]},"apps":[],"jobName":"paragraph_1529915839199_-1678474953","id":"20180427-085254_1614475634","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7245"},{"text":"%md\n\nWe must then initialize the neural network.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We must then initialize the neural network.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839200_-1692710663","id":"20180427-091531_1654516559","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7246"},{"text":"val net = new MultiLayerNetwork(conf);\nnet.init();","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"net: org.deeplearning4j.nn.multilayer.MultiLayerNetwork = org.deeplearning4j.nn.multilayer.MultiLayerNetwork@7611a924\n"}]},"apps":[],"jobName":"paragraph_1529915839200_-1692710663","id":"20180427-085416_2130664426","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7247"},{"text":"%md\n\n### Model Training","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Model Training</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839201_-1693095412","id":"20180427-091609_209488885","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7248"},{"text":"%md\nTo train the model, we use 5 epochs with a for loop and simply call the fit method of the MultiLayerNetwork.","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>To train the model, we use 5 epochs with a for loop and simply call the fit method of the MultiLayerNetwork.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839201_-1693095412","id":"20180427-091612_690496444","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7249"},{"text":"for( epoch <- 1 to 5){\n    println(\"Epoch \"+ epoch);\n    net.fit( train );\n    train.reset();\n}","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"Epoch 1\nEpoch 2\nEpoch 3\nEpoch 4\nEpoch 5\n"}]},"apps":[],"jobName":"paragraph_1529915839202_-1691941165","id":"20180427-085448_34088845","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7250"},{"text":"%md\n### Model Evaluation","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Model Evaluation</h3>\n"}]},"apps":[],"jobName":"paragraph_1529915839202_-1691941165","id":"20180427-091640_1250025060","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7251"},{"text":"%md\nWe will now evaluate our trained model. Note that we will use the area under the curve (AUC) metric of the ROC curve.  ","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We will now evaluate our trained model. Note that we will use the area under the curve (AUC) metric of the ROC curve.</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839202_-1691941165","id":"20180427-092017_454140891","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7252"},{"text":"// Evaluate the model\n\nval roc = new ROC(100);\n\nwhile(test.hasNext()){\n    val next = test.next();\n    val features = next.getFeatures();\n    val output = net.output(features);\n    roc.evalTimeSeries(next.getLabels(), output);\n}\nprintln(roc.calculateAUC());","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"roc: org.deeplearning4j.eval.ROC = ROC(thresholdSteps=100, countActualPositive=0, countActualNegative=0, counts={0.0=ROC.CountsForThreshold(threshold=0.0, countTruePositive=0, countFalsePositive=0), 0.01=ROC.CountsForThreshold(threshold=0.01, countTruePositive=0, countFalsePositive=0), 0.02=ROC.CountsForThreshold(threshold=0.02, countTruePositive=0, countFalsePositive=0), 0.03=ROC.CountsForThreshold(threshold=0.03, countTruePositive=0, countFalsePositive=0), 0.04=ROC.CountsForThreshold(threshold=0.04, countTruePositive=0, countFalsePositive=0), 0.05=ROC.CountsForThreshold(threshold=0.05, countTruePositive=0, countFalsePositive=0), 0.06=ROC.CountsForThreshold(threshold=0.06, countTruePositive=0, countFalsePositive=0), 0.07=ROC.CountsForThreshold(threshold=0.07, countTruePositive=0, count...0.6418342941582758\n"}]},"apps":[],"jobName":"paragraph_1529915839203_-1692325914","id":"20180427-203408_907330565","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7253"},{"text":"%md\n\nWe achieve a AUC of 0.64!","dateUpdated":"2018-06-25T08:37:19+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We achieve a AUC of 0.64!</p>\n"}]},"apps":[],"jobName":"paragraph_1529915839203_-1692325914","id":"20180427-092829_1280525034","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7254"},{"text":"%md\n","dateUpdated":"2018-06-25T08:37:19+0000","config":{"colWidth":12,"editorMode":"ace/mode/markdown","results":{},"enabled":true,"editorSetting":{"language":"markdown","editOnDblClick":true}},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1529915839204_-1694249658","id":"20180427-093020_2025344795","dateCreated":"2018-06-25T08:37:19+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:7255"}],"name":"Instacart Single Task","id":"2DFVUCASC","angularObjects":{"2DKTVHEQG:existing_process":[],"2DJ4SFCPD:existing_process":[],"2DJJJ8C1V:existing_process":[],"2DHBWPF6M:existing_process":[],"2DJB51UJ1:existing_process":[],"2DHQZP5Q3:existing_process":[],"2DKS7J9U9:existing_process":[],"2DHPXD7E1:existing_process":[]},"config":{"looknfeel":"default","personalizedMode":"false"},"info":{}}