代码拉取完成，页面将自动刷新
A simple and well designed structure is essential for any Deep Learning project, so after a lot of practice and contributing in tensorflow projects here's a tensorflow project template that combines simplcity, best practice for folder structure and good OOP design. The main idea is that there's much stuff you do every time you start your tensorflow project, so wrapping all this shared stuff will help you to change just the core idea every time you start a new tensorflow project.
So, here's a simple tensorflow template that help you get into your main project faster and just focus on your core (Model, Training, ...etc)
In a nutshell here's how to use this template, so for example assume you want to implement VGG model so you should do the following:
class VGGModel(BaseModel):
def __init__(self, config):
super(VGGModel, self).__init__(config)
#call the build_model and init_saver functions.
self.build_model()
self.init_saver()
def build_model(self):
# here you build the tensorflow graph of any model you want and also define the loss.
pass
def init_saver(self):
# here you initalize the tensorflow saver that will be used in saving the checkpoints.
self.saver = tf.train.Saver(max_to_keep=self.config.max_to_keep)
class VGGTrainer(BaseTrain):
def __init__(self, sess, model, data, config, logger):
super(VGGTrainer, self).__init__(sess, model, data, config, logger)
def train_epoch(self):
"""
implement the logic of epoch:
-loop on the number of iterations in the config and call the train step
-add any summaries you want using the summary
"""
pass
def train_step(self):
"""
implement the logic of the train step
- run the tensorflow session
- return any metrics you need to summarize
"""
pass
sess = tf.Session()
# create instance of the model you want
model = VGGModel(config)
# create your data generator
data = DataGenerator(config)
# create tensorboard logger
logger = Logger(sess, config)
trainer = VGGTrainer(sess, model, data, config, logger)
# here you train your model
trainer.train()
You will find a template file and a simple example in the model and trainer folder that shows you how to try your first model simply.
├── base
│ ├── base_model.py - this file contains the abstract class of the model.
│ └── base_train.py - this file contains the abstract class of the trainer.
│
│
├── model - this folder contains any model of your project.
│ └── example_model.py
│
│
├── trainer - this folder contains trainers of your project.
│ └── example_trainer.py
│
├── mains - here's the main(s) of your project (you may need more than one main).
│ └── example_main.py - here's an example of main that is responsible for the whole pipeline.
│
├── data _loader
│ └── data_generator.py - here's the data_generator that is responsible for all data handling.
│
└── utils
├── logger.py
└── any_other_utils_you_need
Base model is an abstract class that must be Inherited by any model you create, the idea behind this is that there's much shared stuff between all models. The base model contains:
Here's where you implement your model. So you should :
Base trainer is an abstract class that just wrap the training process.
Here's what you should implement in your trainer.
This class is responsible for all data handling and processing and provide an easy interface that can be used by the trainer.
This class is responsible for the tensorboard summary, in your trainer create a dictionary of all tensorflow variables you want to summarize then pass this dictionary to logger.summarize().
This class also supports reporting to Comet.ml which allows you to see all your hyper-params, metrics, graphs, dependencies and more including real-time metric. Add your API key in the configuration file:
For example: "comet_api_key": "your key here"
This template also supports reporting to Comet.ml which allows you to see all your hyper-params, metrics, graphs, dependencies and more including real-time metric.
Add your API key in the configuration file:
For example: "comet_api_key": "your key here"
Here's how it looks after you start training:
You can also link your Github repository to your comet.ml project for full version control. Here's a live page showing the example from this repo
I use Json as configuration method and then parse it, so write all configs you want then parse it using "utils/config/process_config" and pass this configuration object to all other objects.
Here's where you combine all previous part.
Any kind of enhancement or contribution is welcomed.
Thanks for my colleague Mo'men Abdelrazek for contributing in this work. and thanks for Mohamed Zahran for the review. Thanks for Jtoy for including the repo in Awesome Tensorflow.
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
马建仓 AI 助手