Journals - MOST Wiedzy

Logo

A DEEP Q-LEARNING NETWORK FOR SHIP STOWAGE PLANNING PROBLEM

Abstract

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.

Keywords:

Deep Q-Leaning Network (DQN), Container terminal, Ship stowage plan, Markov decision process, Value function approximation, Generalization

Details

Issue
Vol. 24 No. S3(95) (2017)
Section
Latest Articles
Published
22-11-2017
DOI:
https://doi.org/10.1515/pomr-2017-0111
Licencja:
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Open Access License

This journal provides immediate open access to its content under the Creative Commons BY 4.0 license. Authors who publish with this journal retain all copyrights and agree to the terms of the CC BY 4.0 license.

 

Authors

  • Yifan Shen

    Shanghai Maritime University, Scientific Research Academy
  • Ning Zhao

    Shanghai Maritime University, Logistics Engineering College
  • Mengjue Xia

    Shanghai Maritime University, Scientific Research Academy
  • Xueqiang Du

    Shanghai Maritime University, Logistics Engineering College

Download paper