tf.parse_example.features для классификации фотографий с несколькими метками

Question

Я пытаюсь применить код учебника от cloudml-samples/flowers/на наборе фотографий с несколькими метками. Окружающая среда - это Google Cloud Shell. «предварительная обработка» всех учебных и оценочных заданий. Ошибка при запуске учебной задачи.

Я вызвал trainer.task через python и вернул ниже сообщения об ошибках. Пожалуйста, дайте мне знать, если информация о журнале (только какая-то общая информация) с помощью команды gcloud beta ml или что-то еще будет полезно.

Заранее благодарим за внимание.

python -m trainer.task \ 
    --output_path gs://yelp_restaurant_photo_classification/yelp_restaurant_photo_classification/training \ 
    --eval_data_paths gs://yelp_restaurant_photo_classification/yelp_restaurant_photo_classification/preproc/eval* \ 
    --train_data_paths gs://yelp_restaurant_photo_classification/yelp_restaurant_photo_classification/preproc/train* 

INFO:root:Original job data: {} 
INFO:root:setting eval batch size to 100 
INFO:tensorflow:global_step/sec: 0 
INFO:tensorflow:global_step/sec: 0 
W tensorflow/core/framework/op_kernel.cc:968] Invalid argument: Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors.InvalidArgumentError'>, Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExample/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

Caused by op u'inputs/ParseExample/ParseExample', defined at: 
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main 
    "__main__", fname, loader, pkg_name) 
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code 
    exec code in run_globals 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 559, in <module> 
    tf.app.run() 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run 
    sys.exit(main(sys.argv[:1] + flags_passthrough)) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 322, in main 
    run(model, argv) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 453, in run 
    dispatch(args, model, cluster, task) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 494, in dispatch 
    Trainer(args, model, cluster, task).run_training() 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 193, in run_training 
    self.args.batch_size) 
    File "trainer/model.py", line 278, in build_train_graph 
    return self.build_graph(data_paths, batch_size, GraphMod.TRAIN) 
    File "trainer/model.py", line 232, in build_graph 
    parsed = tf.parse_example(tensors.examples, features=feature_map) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 307, in parse_example 
    dense_types, dense_defaults, dense_shapes, name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 405, in _parse_example_raw 
    name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_parsing_ops.py", line 165, in _parse_example 
    dense_shapes=dense_shapes, name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op 
    op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2380, in create_op 
    original_op=self._default_original_op, op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1298, in __init__ 
    self._traceback = _extract_stack() 

InvalidArgumentError (see above for traceback): Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExample/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors.InvalidArgumentError'>, Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExample/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

Caused by op u'inputs/ParseExample/ParseExample', defined at: 
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main 
    "__main__", fname, loader, pkg_name) 
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code 
    exec code in run_globals 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 559, in <module> 
    tf.app.run() 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run 
    sys.exit(main(sys.argv[:1] + flags_passthrough)) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 322, in main 
    run(model, argv) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 453, in run 
    dispatch(args, model, cluster, task) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 494, in dispatch 
    Trainer(args, model, cluster, task).run_training() 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 193, in run_training 
    self.args.batch_size) 
    File "trainer/model.py", line 278, in build_train_graph 
    return self.build_graph(data_paths, batch_size, GraphMod.TRAIN) 
    File "trainer/model.py", line 232, in build_graph 
    parsed = tf.parse_example(tensors.examples, features=feature_map) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 307, in parse_example 
    dense_types, dense_defaults, dense_shapes, name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 405, in _parse_example_raw 
    name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_parsing_ops.py", line 165, in _parse_example 
    dense_shapes=dense_shapes, name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op 
    op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2380, in create_op 
    original_op=self._default_original_op, op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1298, in __init__ 
    self._traceback = _extract_stack() 

InvalidArgumentError (see above for traceback): Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExample/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

Traceback (most recent call last): 
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main 
    "__main__", fname, loader, pkg_name) 
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code 
    exec code in run_globals 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 559, in <module> 
    tf.app.run() 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run 
    sys.exit(main(sys.argv[:1] + flags_passthrough)) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 322, in main 
    run(model, argv) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 453, in run 
    dispatch(args, model, cluster, task) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 494, in dispatch 
    Trainer(args, model, cluster, task).run_training() 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 245, in run_training 
    self.global_step = session.run(to_run)[0] 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run 
    run_metadata_ptr) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 915, in _run 
    feed_dict_string, options, run_metadata) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _do_run 
    target_list, options, run_metadata) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 985, in _do_call 
    raise type(e)(node_def, op, message) 
tensorflow.python.framework.errors.InvalidArgumentError: Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but output shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExample/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

Caused by op u'inputs/ParseExample/ParseExample', defined at: 
    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main 
    "__main__", fname, loader, pkg_name) 
    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code 
    exec code in run_globals 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 559, in <module> 
    tf.app.run() 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run 
    sys.exit(main(sys.argv[:1] + flags_passthrough)) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 322, in main 
    run(model, argv) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 453, in run 
    dispatch(args, model, cluster, task) 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 494, in dispatch 
    Trainer(args, model, cluster, task).run_training() 
    File "/home/slalomconsultingsf/git/cloudml-samples/yelp_restaurant_photo_classification/trainer/task.py", line 193, in run_training 
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors.InvalidArgumentError'>, Name: <unknown>, Key: label, Index: 
    self.args.batch_size) 
    File "trainer/model.py", line 278, in build_train_graph 
    return self.build_graph(data_paths, batch_size, GraphMod.TRAIN) 
    File "trainer/model.py", line 232, in build_graph 
    parsed = tf.parse_example(tensors.examples, features=feature_map) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 307, in parse_example 
    dense_types, dense_defaults, dense_shapes, name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/parsing_ops.py", line 405, in _parse_example_raw 
    name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_parsing_ops.py", line 165, in _parse_example 
    dense_shapes=dense_shapes, name=name) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op 
    op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2380, in create_op 
    original_op=self._default_original_op, op_def=op_def) 
    File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1298, in __init__ 
    self._traceback = _extract_stack() 
InvalidArgumentError (see above for traceback): Name: <unknown>, Key: label, Index: 0. Number of int64 values != expected. Values size: 5 but outpu 
t shape: [1] 
     [[Node: inputs/ParseExample/ParseExample = ParseExample[Ndense=3, Nsparse=0, Tdense=[DT_FLOAT, DT_STRING, DT_INT64], dense_shapes=[[2048], [ 
], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch:1, inputs/ParseExample/ParseExample/names, inputs/ParseExamp 
le/ParseExample/dense_keys_0, inputs/ParseExample/ParseExample/dense_keys_1, inputs/ParseExample/ParseExample/dense_keys_2, inputs/ParseExample/Const 
, inputs/ParseExample/Reshape, inputs/ParseExample/Reshape_1)]] 

Answer 1

Образец цветов непосредственно не поддерживает классификацию с несколькими метками. Точная проблема в вашем коде, я считаю, связана с shape=[1] in model.py.

Удовлетворение примерами с несколькими ярлыками не будет работать из коробки с этим кодом, который был разработан, чтобы быть хорошим примером того, как начать работу со сложным рабочим процессом (т.е. восстановление с контрольных точек, передача обучения, встраивание в Dataflow и т. Д.), Более того, чем образный швейцарский армейский нож.

Должно быть только несколько строк, которые вы должны были бы внести изменения в model.py, чтобы работать с вашим примером нескольких ярлыков. Кроме того, я уверен, что другие члены сообщества будут очень признательны за любой прогресс, который вы сделаете, поэтому, пожалуйста, подумайте о создании pull request, если вы получите образец, который выполняет как многократные, так и однократные проблемы!