{"paragraphs":[{"text":"%md\n### Note\n\nPlease view the [README](https://github.com/eclipse/deeplearning4j-examples/blob/master/tutorials/README.md) to learn about installing, setting up dependencies, and importing notebooks in Zeppelin","dateUpdated":"2018-01-17T19:30:54+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Note</h3>\n<p>Please view the <a href=\"https://github.com/eclipse/deeplearning4j-examples/blob/master/tutorials/README.md\">README</a> to learn about installing, setting up dependencies, and importing notebooks in Zeppelin</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445575_-1890608365","id":"20180115-062124_1656190287","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:2705","user":"anonymous","dateFinished":"2018-01-17T19:31:26+0000","dateStarted":"2018-01-17T19:31:26+0000"},{"text":"%md\n### Background","dateUpdated":"2018-01-17T19:30:56+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Background</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445580_-1888609047","id":"20180116-040712_1768508310","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2706","user":"anonymous","dateFinished":"2018-01-17T19:31:26+0000","dateStarted":"2018-01-17T19:31:26+0000"},{"text":"%md\nIn this tutorial, we will learn how to apply a long-short term memory (LSTM) neural network to a medical time series problem. The data used comes from 4000 intensive care unit (ICU) patients and the goal is to predict the mortality of patients using 6 general descriptor features, such as age, gender, and weight along with 37 sequential features, such as cholesterol level, temperature, pH, and glucose level. Each patient has multiple measurements of the sequential features, with patients having a different amount of measurements taken. Furthermore, the time between measurements also differ among patients as well. \n\nA LSTM is well suited for this type of problem due to the sequential nature of the data. In addition, LSTM networks avoid vanishing and exploding gradients and are able to effectively capture long term dependencies due to its cell state, a feature not present in typical recurrent networks. For a more in depth explanation of LSTM's, see https://deeplearning4j.org/lstm.html.","dateUpdated":"2018-01-17T19:30:57+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>In this tutorial, we will learn how to apply a long-short term memory (LSTM) neural network to a medical time series problem. The data used comes from 4000 intensive care unit (ICU) patients and the goal is to predict the mortality of patients using 6 general descriptor features, such as age, gender, and weight along with 37 sequential features, such as cholesterol level, temperature, pH, and glucose level. Each patient has multiple measurements of the sequential features, with patients having a different amount of measurements taken. Furthermore, the time between measurements also differ among patients as well. </p>\n<p>A LSTM is well suited for this type of problem due to the sequential nature of the data. In addition, LSTM networks avoid vanishing and exploding gradients and are able to effectively capture long term dependencies due to its cell state, a feature not present in typical recurrent networks. For a more in depth explanation of LSTM&rsquo;s, see <a href=\"https://deeplearning4j.org/lstm.html\">https://deeplearning4j.org/lstm.html</a>.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445580_-419624708","id":"20180116-040741_1071141793","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2707","user":"anonymous","dateFinished":"2018-01-17T19:31:27+0000","dateStarted":"2018-01-17T19:31:27+0000"},{"text":"%md\n### Imports","dateUpdated":"2018-01-17T19:30:59+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Imports</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445581_-1512572081","id":"20180115-062155_892306848","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2708","user":"anonymous","dateFinished":"2018-01-17T19:31:27+0000","dateStarted":"2018-01-17T19:31:27+0000"},{"text":"import org.datavec.api.records.reader.SequenceRecordReader;\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader;\nimport org.datavec.api.split.NumberedFileInputSplit;\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator;\nimport org.deeplearning4j.eval.ROC;\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm;\nimport org.deeplearning4j.nn.conf.ComputationGraphConfiguration;\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration;\nimport org.deeplearning4j.nn.conf.Updater;\nimport org.deeplearning4j.nn.conf.layers.GravesLSTM;\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer;\nimport org.deeplearning4j.nn.graph.ComputationGraph;\nimport org.deeplearning4j.optimize.listeners.ScoreIterationListener;\nimport org.nd4j.linalg.api.ndarray.INDArray;\nimport org.deeplearning4j.nn.weights.WeightInit;\nimport org.nd4j.linalg.activations.Activation;\nimport org.nd4j.linalg.dataset.api.DataSet;\nimport org.nd4j.linalg.lossfunctions.LossFunctions;\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator;\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\nimport java.io.File;\nimport org.apache.commons.io.FileUtils;\nimport org.apache.commons.io.FilenameUtils;\nimport java.io.IOException;\nimport java.util.HashMap;\nimport java.util.Arrays;\nimport java.net.URL;\nimport java.io.BufferedInputStream;\nimport java.io.FileInputStream;\nimport java.io.BufferedOutputStream;\nimport java.io.FileOutputStream;\nimport java.lang.Byte;\n\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry;\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream;\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;\n\n","dateUpdated":"2018-01-17T19:31:01+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"import org.datavec.api.records.reader.SequenceRecordReader\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader\nimport org.datavec.api.split.NumberedFileInputSplit\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator\nimport org.deeplearning4j.eval.ROC\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.ComputationGraphConfiguration\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration\nimport org.deeplearning4j.nn.conf.Updater\nimport org.deeplearning4j.nn.conf.layers.GravesLSTM\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer\nimport org.deeplearning4j.nn.graph.ComputationGraph\nimport org.deeplearning4j.optimize.listeners.ScoreIterationListener\nimport org.nd4j.linalg.api.ndarray.INDArray\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.dataset.api.DataSet\nimport org.nd4j.linalg.lossfunctions.LossFunctions\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator\nimport org.slf4j.Logger\nimport org.slf4j.LoggerFactory\nimport java.io.File\nimport org.apache.commons.io.FileUtils\nimport org.apache.commons.io.FilenameUtils\nimport java.io.IOException\nimport java.util.HashMap\nimport java.util.Arrays\nimport java.net.URL\nimport java.io.BufferedInputStream\nimport java.io.FileInputStream\nimport java.io.BufferedOutputStream\nimport java.io.FileOutputStream\nimport java.lang.Byte\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream\n"}]},"apps":[],"jobName":"paragraph_1516217445582_-2052542973","id":"20180115-062209_1381660770","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2709","user":"anonymous","dateFinished":"2018-01-17T19:31:32+0000","dateStarted":"2018-01-17T19:31:27+0000"},{"text":"%md\n\nNow that we have imported everything needed to run this tutorial, we will start with obtaining the data and then converting the  data into a format a neural network can understand. ","dateUpdated":"2018-01-17T19:31:03+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Now that we have imported everything needed to run this tutorial, we will start with obtaining the data and then converting the data into a format a neural network can understand.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445583_1479312588","id":"20180116-041926_21274632","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2710","user":"anonymous","dateFinished":"2018-01-17T19:31:32+0000","dateStarted":"2018-01-17T19:31:32+0000"},{"text":"%md\n### Data Source","dateUpdated":"2018-01-17T19:31:04+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1516217445583_-235105107","id":"20180115-063620_1089188478","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:2711","user":"anonymous","dateFinished":"2018-01-17T19:31:33+0000","dateStarted":"2018-01-17T19:31:33+0000"},{"text":"%md\nThe data is contained in a compressed tar.gz file. We will have to download the data from the url below and then extract csv files containing the ICU data. Each patient will have a separate csv file for the features and labels. The features will be contained in a directory called sequence and the labels will be contained in a directory called mortality. The features are contained in a single csv file with the columns representing the features and the rows representing different time steps. The labels are contained in a single csv file which contains a value of 0 indicating death and a value of 1 indicating survival.","dateUpdated":"2018-01-17T19:31:06+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>The data is contained in a compressed tar.gz file. We will have to download the data from the url below and then extract csv files containing the ICU data. Each patient will have a separate csv file for the features and labels. The features will be contained in a directory called sequence and the labels will be contained in a directory called mortality. The features are contained in a single csv file with the columns representing the features and the rows representing different time steps. The labels are contained in a single csv file which contains a value of 0 indicating death and a value of 1 indicating survival.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445585_-112508991","id":"20180116-042022_82417057","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2712","user":"anonymous","dateFinished":"2018-01-17T19:31:33+0000","dateStarted":"2018-01-17T19:31:33+0000"},{"text":"val DATA_URL = \"https://skymindacademy.blob.core.windows.net/physionet2012/physionet2012.tar.gz\"\nval DATA_PATH = FilenameUtils.concat(System.getProperty(\"java.io.tmpdir\"), \"dl4j_physionet/\")","dateUpdated":"2018-01-17T19:31:08+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"DATA_URL: String = https://skymindacademy.blob.core.windows.net/physionet2012/physionet2012.tar.gz\nDATA_PATH: String = /tmp/dl4j_physionet/\n"}]},"apps":[],"jobName":"paragraph_1516217445586_306879171","id":"20180115-063627_2071788954","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2713","user":"anonymous","dateFinished":"2018-01-17T19:31:33+0000","dateStarted":"2018-01-17T19:31:33+0000"},{"text":"%md\n### Download Data","dateUpdated":"2018-01-17T19:31:09+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Download Data</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445588_2098553898","id":"20180115-063817_834797484","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2714","user":"anonymous","dateFinished":"2018-01-17T19:31:33+0000","dateStarted":"2018-01-17T19:31:33+0000"},{"text":"%md\nTo download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.","dateUpdated":"2018-01-17T19:31:39+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>To download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445590_-2139649544","id":"20180116-042108_1540744847","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2715","user":"anonymous","dateFinished":"2018-01-17T19:31:39+0000","dateStarted":"2018-01-17T19:31:39+0000"},{"text":"val directory = new File(DATA_PATH)\ndirectory.mkdir() // create new directory at specified path\n\nval archizePath = DATA_PATH + \"physionet2012.tar.gz\" // set path for tar.gz file\nval archiveFile = new File(archizePath) // create tar.gz file\nval extractedPath = DATA_PATH + \"physionet2012\" \nval extractedFile = new File(extractedPath)\n\nFileUtils.copyURLToFile(new URL(DATA_URL), archiveFile) // copy data from URL to file","dateUpdated":"2018-01-17T19:31:50+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"directory: java.io.File = /tmp/dl4j_physionet\nres0: Boolean = true\narchizePath: String = /tmp/dl4j_physionet/physionet2012.tar.gz\narchiveFile: java.io.File = /tmp/dl4j_physionet/physionet2012.tar.gz\nextractedPath: String = /tmp/dl4j_physionet/physionet2012\nextractedFile: java.io.File = /tmp/dl4j_physionet/physionet2012\n"}]},"apps":[],"jobName":"paragraph_1516217445591_1243316908","id":"20180115-063829_1644657611","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2716","user":"anonymous","dateFinished":"2018-01-17T19:32:00+0000","dateStarted":"2018-01-17T19:31:50+0000"},{"text":"%md \n\nNext, we must extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directory, and copy the files into our temporary directory. ","dateUpdated":"2018-01-17T19:31:53+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Next, we must extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directory, and copy the files into our temporary directory.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445591_-635008405","id":"20180115-064020_988667885","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2717","user":"anonymous","dateFinished":"2018-01-17T19:32:00+0000","dateStarted":"2018-01-17T19:32:00+0000"},{"text":"var fileCount = 0\nvar dirCount = 0\nval BUFFER_SIZE = 4096\n\nval tais = new TarArchiveInputStream(new GzipCompressorInputStream( new BufferedInputStream( new FileInputStream(archizePath))))\n\nvar entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n\nwhile(entry != null){\n    if (entry.isDirectory()) {\n        new File(DATA_PATH + entry.getName()).mkdirs()\n        dirCount = dirCount + 1\n        fileCount = 0\n    }\n    else {\n        \n        val data = new Array[scala.Byte](4 * BUFFER_SIZE)\n\n        val fos = new FileOutputStream(DATA_PATH + entry.getName());\n        val dest = new BufferedOutputStream(fos, BUFFER_SIZE);\n        var count = tais.read(data, 0, BUFFER_SIZE)\n        \n        while (count != -1) {\n            dest.write(data, 0, count)\n            count = tais.read(data, 0, BUFFER_SIZE)\n        }\n        \n        dest.close()\n        fileCount = fileCount + 1\n    }\n    if(fileCount % 1000 == 0){\n        print(\".\")\n    }\n    \n    entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n}","dateUpdated":"2018-01-17T19:38:41+0000","config":{"tableHide":true,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"fileCount: Int = 0\ndirCount: Int = 0\nBUFFER_SIZE: Int = 4096\ntais: org.apache.commons.compress.archivers.tar.TarArchiveInputStream = org.apache.commons.compress.archivers.tar.TarArchiveInputStream@5e55361\nentry: org.apache.commons.compress.archivers.tar.TarArchiveEntry = org.apache.commons.compress.archivers.tar.TarArchiveEntry@221cf6b\n........................................."}]},"apps":[],"jobName":"paragraph_1516217445591_1947002165","id":"20180115-070134_320419844","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2718","user":"anonymous","dateFinished":"2018-01-17T19:32:15+0000","dateStarted":"2018-01-17T19:32:00+0000"},{"text":"%md\n### DataSetIterators","dateUpdated":"2018-01-17T19:31:56+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>DataSetIterators</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445592_312229906","id":"20180115-070220_1700945870","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2719","user":"anonymous","dateFinished":"2018-01-17T19:32:15+0000","dateStarted":"2018-01-17T19:32:15+0000"},{"text":"%md\nOur next goal is to convert the raw data (csv files) into a DataSetIterator, which can then be fed into a neural network for training. Our training data will have 3200 examples which will be represented by a single DataSetIterator, and the testing data will have 800 examples which will be represented by a separate DataSet Iterator.","dateUpdated":"2018-01-17T19:32:00+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Our next goal is to convert the raw data (csv files) into a DataSetIterator, which can then be fed into a neural network for training. Our training data will have 3200 examples which will be represented by a single DataSetIterator, and the testing data will have 800 examples which will be represented by a separate DataSet Iterator.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445593_1974034268","id":"20180116-043157_768466617","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2720","user":"anonymous","dateFinished":"2018-01-17T19:32:15+0000","dateStarted":"2018-01-17T19:32:15+0000"},{"text":"val NB_TRAIN_EXAMPLES = 3200 // number of training examples\nval NB_TEST_EXAMPLES = 800 // number of testing examples","dateUpdated":"2018-01-17T19:32:04+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"NB_TRAIN_EXAMPLES: Int = 3200\nNB_TEST_EXAMPLES: Int = 800\n"}]},"apps":[],"jobName":"paragraph_1516217445593_309076666","id":"20180115-071757_4986863","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2721","user":"anonymous","dateFinished":"2018-01-17T19:32:19+0000","dateStarted":"2018-01-17T19:32:15+0000"},{"text":"%md\nIn order to obtain DataSetIterators, we must first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. We will first set the directories for the features and labels and initialize the CSVSequenceRecordReaders.\n\nNext, we can initialize the SequenceRecordReaderDataSetIterator using the previously created CSVSequenceRecordReaders. We will use an alignment mode of ALIGN_END. This alignment mode is needed due to the fact that the number of time steps differs between different patients. Because the mortality label is always at the end of the sequence, we need all the sequences aligned so that the time step with the mortality label is the last time step for all patients. For a more in depth explanation of alignment modes, see https://deeplearning4j.org/usingrnns. ","dateUpdated":"2018-01-17T19:32:05+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>In order to obtain DataSetIterators, we must first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. We will first set the directories for the features and labels and initialize the CSVSequenceRecordReaders.</p>\n<p>Next, we can initialize the SequenceRecordReaderDataSetIterator using the previously created CSVSequenceRecordReaders. We will use an alignment mode of ALIGN_END. This alignment mode is needed due to the fact that the number of time steps differs between different patients. Because the mortality label is always at the end of the sequence, we need all the sequences aligned so that the time step with the mortality label is the last time step for all patients. For a more in depth explanation of alignment modes, see <a href=\"https://deeplearning4j.org/usingrnns\">https://deeplearning4j.org/usingrnns</a>.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445594_-38903231","id":"20180115-070711_769734985","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2722","user":"anonymous","dateFinished":"2018-01-17T19:32:19+0000","dateStarted":"2018-01-17T19:32:19+0000"},{"text":"val path = FilenameUtils.concat(DATA_PATH, \"physionet2012/\") // set parent directory\n\nval featureBaseDir = FilenameUtils.concat(path, \"sequence\") // set feature directory\nval mortalityBaseDir = FilenameUtils.concat(path, \"mortality\") // set label directory\n\n// Load training data\n\nval trainFeatures = new CSVSequenceRecordReader(1, \",\")\ntrainFeatures.initialize( new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", 0, NB_TRAIN_EXAMPLES - 1))\n\nval trainLabels = new CSVSequenceRecordReader()\ntrainLabels.initialize(new NumberedFileInputSplit(mortalityBaseDir + \"/%d.csv\", 0, NB_TRAIN_EXAMPLES - 1))\n\nval trainData = new SequenceRecordReaderDataSetIterator(trainFeatures, trainLabels,\n              32, 2, false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END)\n              \n              \n// Load testing data\nval testFeatures = new CSVSequenceRecordReader(1, \",\");\ntestFeatures.initialize(new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", NB_TRAIN_EXAMPLES, NB_TRAIN_EXAMPLES + NB_TEST_EXAMPLES - 1));\n       \nval testLabels = new CSVSequenceRecordReader();\ntestLabels.initialize(new NumberedFileInputSplit(mortalityBaseDir + \"/%d.csv\", NB_TRAIN_EXAMPLES, NB_TRAIN_EXAMPLES  + NB_TEST_EXAMPLES - 1));\n\nval testData = new SequenceRecordReaderDataSetIterator(testFeatures, testLabels,\n                32, 2, false,SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);","dateUpdated":"2018-01-17T19:39:30+0000","config":{"tableHide":true,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"path: String = /tmp/dl4j_physionet/physionet2012/\nfeatureBaseDir: String = /tmp/dl4j_physionet/physionet2012/sequence\nmortalityBaseDir: String = /tmp/dl4j_physionet/physionet2012/mortality\nfileFeature: java.io.File = /tmp/dl4j_physionet/physionet2012/sequence\nfileMortality: java.io.File = /tmp/dl4j_physionet/physionet2012/mortality\ntrainFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@467e1cb4\ntrainLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@3059dd00\ntrainData: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@43185e51\ntestFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@3b22a193\ntestLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@2549870c\ntestData: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@2dc47fb7\n"}]},"apps":[],"jobName":"paragraph_1516217445595_2088178641","id":"20180115-070257_386614995","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2723","user":"anonymous","dateFinished":"2018-01-17T19:32:22+0000","dateStarted":"2018-01-17T19:32:19+0000"},{"text":"%md\n### Neural Network Configuration","dateUpdated":"2018-01-17T19:32:17+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Neural Network Configuration</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445595_-733251238","id":"20180115-062914_1963273192","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2724","user":"anonymous","dateFinished":"2018-01-17T19:32:23+0000","dateStarted":"2018-01-17T19:32:23+0000"},{"text":"%md\nNow we can finally configure and then initialize the neural network for this problem. We will be using the ComputationGraph class of DL4J.","dateUpdated":"2018-01-17T19:32:24+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Now we can finally configure and then initialize the neural network for this problem. We will be using the ComputationGraph class of DL4J.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445596_-1762341852","id":"20180116-045323_1278918662","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2725","user":"anonymous","dateFinished":"2018-01-17T19:32:24+0000","dateStarted":"2018-01-17T19:32:24+0000"},{"text":"// Set neural network parameters\nval NB_INPUTS = 86\nval NB_EPOCHS = 10\nval RANDOM_SEED = 1234\nval LEARNING_RATE = 0.005\nval BATCH_SIZE = 32\nval LSTM_LAYER_SIZE = 200\nval NUM_LABEL_CLASSES = 2 ","dateUpdated":"2018-01-17T19:32:28+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"NB_INPUTS: Int = 86\nNB_EPOCHS: Int = 10\nRANDOM_SEED: Int = 1234\nLEARNING_RATE: Double = 0.005\nBATCH_SIZE: Int = 32\nLSTM_LAYER_SIZE: Int = 200\nNUM_LABEL_CLASSES: Int = 2\n"}]},"apps":[],"jobName":"paragraph_1516217445597_-1164540157","id":"20180115-062305_1051366040","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2726","user":"anonymous","dateFinished":"2018-01-17T19:32:29+0000","dateStarted":"2018-01-17T19:32:28+0000"},{"text":"val conf = new NeuralNetConfiguration.Builder()\n        .seed(RANDOM_SEED)\n        .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n        .learningRate(LEARNING_RATE)\n        .weightInit(WeightInit.XAVIER)\n        .updater(Updater.ADAM)\n        .dropOut(0.25)\n        .graphBuilder()\n        .addInputs(\"trainFeatures\")\n        .setOutputs(\"predictMortality\")\n        .addLayer(\"L1\", new GravesLSTM.Builder()\n                .nIn(NB_INPUTS)\n                .nOut(LSTM_LAYER_SIZE)\n                .forgetGateBiasInit(1)\n                .activation(Activation.TANH)\n                .build(),\n                \"trainFeatures\")\n        .addLayer(\"predictMortality\", new RnnOutputLayer.Builder(LossFunctions.LossFunction.XENT)\n                .activation(Activation.SOFTMAX)\n                .nIn(LSTM_LAYER_SIZE).nOut(NUM_LABEL_CLASSES).build(),\"L1\")\n        .pretrain(false).backprop(true)\n        .build()\n        \nval model = new ComputationGraph(conf)","dateUpdated":"2018-01-17T19:32:47+0000","config":{"tableHide":true,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"conf: org.deeplearning4j.nn.conf.ComputationGraphConfiguration =\n{\n  \"backprop\" : true,\n  \"backpropType\" : \"Standard\",\n  \"cacheMode\" : \"NONE\",\n  \"defaultConfiguration\" : {\n    \"cacheMode\" : \"NONE\",\n    \"iterationCount\" : 0,\n    \"l1ByParam\" : { },\n    \"l2ByParam\" : { },\n    \"layer\" : null,\n    \"leakyreluAlpha\" : 0.0,\n    \"learningRateByParam\" : { },\n    \"learningRatePolicy\" : \"None\",\n    \"lrPolicyDecayRate\" : \"NaN\",\n    \"lrPolicyPower\" : \"NaN\",\n    \"lrPolicySteps\" : \"NaN\",\n    \"maxNumLineSearchIterations\" : 5,\n    \"miniBatch\" : true,\n    \"minimize\" : true,\n    \"numIterations\" : 1,\n    \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n    \"pretrain\" : false,\n    \"seed\" : 1234,\n    \"stepFunction\" : null,\n    \"useDropConnect\" : false,\n    \"useRegularization\" : false,\n    \"variables\" : [ ]...model: org.deeplearning4j.nn.graph.ComputationGraph = org.deeplearning4j.nn.graph.ComputationGraph@d7846d3\n"}]},"apps":[],"jobName":"paragraph_1516217445599_-2004714755","id":"20180115-072202_955845730","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2727","user":"anonymous","dateFinished":"2018-01-17T19:33:14+0000","dateStarted":"2018-01-17T19:32:31+0000"},{"text":"%md\n\n### Training","dateUpdated":"2018-01-17T19:32:50+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Training</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445599_249732777","id":"20180115-200258_1843146082","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2728","user":"anonymous","dateFinished":"2018-01-17T19:33:14+0000","dateStarted":"2018-01-17T19:33:14+0000"},{"text":"%md\nTo train the neural network, we simply call the fit method of the ComputationGraph on the trainData DataSetIterator within a for loop.","dateUpdated":"2018-01-17T19:32:51+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>To train the neural network, we simply call the fit method of the ComputationGraph on the trainData DataSetIterator within a for loop.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445601_-844872833","id":"20180116-045440_285367027","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2729","user":"anonymous","dateFinished":"2018-01-17T19:33:14+0000","dateStarted":"2018-01-17T19:33:14+0000"},{"text":"for( a <- 1 to 2){\n    model.fit(trainData)\n    trainData.reset()\n}","dateUpdated":"2018-01-17T19:33:41+0000","config":{"tableHide":true,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1516217445603_-652449116","id":"20180115-201008_417710019","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2730","user":"anonymous","dateFinished":"2018-01-17T19:38:17+0000","dateStarted":"2018-01-17T19:33:33+0000","results":{"code":"SUCCESS","msg":[]}},{"text":"%md\n\n### Model Evaluation","dateUpdated":"2018-01-17T19:40:41+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Model Evaluation</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445604_-787007366","id":"20180115-200622_955491798","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2731","user":"anonymous","dateFinished":"2018-01-17T19:40:42+0000","dateStarted":"2018-01-17T19:40:42+0000"},{"text":"%md\nFinally, we can evaluate the model with the testing split using the AUC (area under the curve metric ) using a ROC curve. A randomly guessing model will have an AUC close to 0.50, while a perfect model will achieve an AUC of 1.00","dateUpdated":"2018-01-17T19:40:12+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Finally, we can evaluate the model with the testing split using the AUC (area under the curve metric ) using a ROC curve. A randomly guessing model will have an AUC close to 0.50, while a perfect model will achieve an AUC of 1.00</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445604_28477177","id":"20180116-045600_2068488543","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2732","user":"anonymous","dateFinished":"2018-01-17T19:40:25+0000","dateStarted":"2018-01-17T19:40:24+0000"},{"text":"val roc = new ROC(100);\n\nwhile (testData.hasNext()) {\n    val batch = testData.next();\n    val output = model.output(batch.getFeatures());\n    roc.evalTimeSeries(batch.getLabels(), output(0));\n}\n\nprintln(\"FINAL TEST AUC: \" + roc.calculateAUC());","dateUpdated":"2018-01-17T19:39:30+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"roc: org.deeplearning4j.eval.ROC = ROC(thresholdSteps=100, countActualPositive=0, countActualNegative=0, counts={0.0=ROC.CountsForThreshold(threshold=0.0, countTruePositive=0, countFalsePositive=0), 0.01=ROC.CountsForThreshold(threshold=0.01, countTruePositive=0, countFalsePositive=0), 0.02=ROC.CountsForThreshold(threshold=0.02, countTruePositive=0, countFalsePositive=0), 0.03=ROC.CountsForThreshold(threshold=0.03, countTruePositive=0, countFalsePositive=0), 0.04=ROC.CountsForThreshold(threshold=0.04, countTruePositive=0, countFalsePositive=0), 0.05=ROC.CountsForThreshold(threshold=0.05, countTruePositive=0, countFalsePositive=0), 0.06=ROC.CountsForThreshold(threshold=0.06, countTruePositive=0, countFalsePositive=0), 0.07=ROC.CountsForThreshold(threshold=0.07, countTruePositive=0, count...FINAL TEST AUC: 0.6877951190417019\n"}]},"apps":[],"jobName":"paragraph_1516217445605_-861903053","id":"20180115-072548_1324690838","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2733","user":"anonymous","dateFinished":"2018-01-17T19:40:23+0000","dateStarted":"2018-01-17T19:39:30+0000"},{"text":"%md\n\nWe see that this model achieves an AUC on the test set of 0.69!","dateUpdated":"2018-01-17T19:40:38+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>We see that this model achieves an AUC on the test set of 0.69!</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1516217445605_-633570850","id":"20180115-204936_1000695921","dateCreated":"2018-01-17T19:30:45+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:2734","user":"anonymous","dateFinished":"2018-01-17T19:40:38+0000","dateStarted":"2018-01-17T19:40:38+0000"},{"text":"%md\n","dateUpdated":"2018-01-17T19:30:45+0000","config":{"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1516217445606_33058463","id":"20180116-045740_857601022","dateCreated":"2018-01-17T19:30:45+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:2735"}],"name":"ClinicalTimeSeriesLSTMExample","id":"2D71WD51K","angularObjects":{"2D6ZDFUHH:shared_process":[],"2D3P4KTQE:shared_process":[],"2D4NXXBXE:shared_process":[],"2D4YCKUSH:shared_process":[],"2D3DY6372:shared_process":[],"2D3F3XCBM:shared_process":[],"2D4VVFM1E:shared_process":[],"2D4HJRZ6A:shared_process":[],"2D3VY1C98:shared_process":[],"2D623FPGE:shared_process":[],"2D38K2X83:shared_process":[],"2D46AX3BS:shared_process":[],"2D3ZDFS7E:shared_process":[],"2D34F9HKR:shared_process":[],"2D6QYUSFB:shared_process":[],"2D4KHXP1T:shared_process":[],"2D4J2N6CD:shared_process":[],"2D6S3TTBE:shared_process":[],"2D5TKWCPG:shared_process":[],"2D42SGVBE:shared_process":[]},"config":{"looknfeel":"default","personalizedMode":"false"},"info":{}}