# minddata_analyser

**Repository Path**: danishfarid/minddata_analyser

## Basic Information

- **Project Name**: minddata_analyser
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 1
- **Created**: 2021-10-02
- **Last Updated**: 2021-10-19

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

Oct 2021 <br/>
Autotune - Prototype <br/> 
Danish / Jaspreet <br/>
Huawei Canda - DD <br/>

This is a small tool designed to allow for rapid prototyping of a paramter tuning setup for running mindspore models. 

FOLDER STRUCTURE
----------------
    1. scripts -> This will contain the running and reset scripts. All sample runs will also be contained in this directory
        -> run_standalone_train_gpu.sh: This is the driver batch file that will create the train folder structure. 
            All relevant files will be contained in a folder called "analysis_run_{time}"
        -> clean_run_logs.sh: Running this will clear all analysis directories
    2. models -> This will contain folders will directories
        1. Model scripts, src and config files
        2. You are required to put a "map_workers.json" to detail:
            for example -> {"num_map_ops": 3} this will be needed for the analyser to give 
            the initial config and setup the following configs for subsequent runs.
    3. config_init.py -> This will setup the initial config for the model, it will depend on the map_workers.json file
    4. config_update.py -> This file contains the logic for reading the log files - and running next config generation logic

RUNNING INSTRUCTIONS
--------------------
    1. The primary running script can be found inside the scripts folder
    2. The primary script can be run by SCRIPT.sh [MODEL DIRECTORY] [DATA DIRECTORY] [CONFIG YAML] [ANALYSIS RUN COUNT]

        SAMPLE COMMAND => "bash run_standalone_train_gpu.sh ../models/resnet /nfsshare/data/cifar10/cifar-10-batches-bin/ resnet_gpu.yaml 3"

    3. Another directory will be created for the analysis data inside a directory called analysis_run_{time}.

CHANGING PROTOTYPE
------------------
    Changing the logic in the config_init and config_update files will change overall functionality.
    
After starting
--------------
    After running the main bash script, you can find the data inside the latest created analysis_run_{time}. The different runs will be contained in sep "train" directories. Each is a self contained model run.