I have been running the train code for hours and it still doesnt start.I use command prompt to train
Here more information: Python 3.10.9 Tensorflow 2.9.0 Tensorflow-gpu 2.9.0 CUDA 11.2 CUDNN 8.1.1
Here is the code to train
python Tensorflow\models\research\object_detection\model_main_tf2.py --model_dir=Tensorflow\workspace\models\my_ssd_mobnet --pipeline_config_path=Tensorflow\workspace\models\my_ssd_mobnet\pipeline.config --num_train_steps=2000 Here is the output given
2024-03-02 18:25:01.644759: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2024-03-02 18:25:02.036429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 2145 MB memory: -> device: 0, name: NVIDIA GeForce GTX 1650 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5 INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',) I0302 18:25:02.112543 25268 mirrored_strategy.py:374] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',) INFO:tensorflow:Maybe overwriting train_steps: 2000 I0302 18:25:02.115335 25268 config_util.py:552] Maybe overwriting train_steps: 2000 INFO:tensorflow:Maybe overwriting use_bfloat16: False I0302 18:25:02.116389 25268 config_util.py:552] Maybe overwriting use_bfloat16: False WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\model_lib_v2.py:563: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function W0302 18:25:02.137276 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\model_lib_v2.py:563: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function INFO:tensorflow:Reading unweighted datasets: ['Tensorflow\\workspace\\annotations\\train.record'] I0302 18:25:02.141191 25268 dataset_builder.py:162] Reading unweighted datasets: ['Tensorflow\\workspace\\annotations\\train.record'] INFO:tensorflow:Reading record datasets for input file: ['Tensorflow\\workspace\\annotations\\train.record'] I0302 18:25:02.142155 25268 dataset_builder.py:79] Reading record datasets for input file: ['Tensorflow\\workspace\\annotations\\train.record'] INFO:tensorflow:Number of filenames to read: 1 I0302 18:25:02.142155 25268 dataset_builder.py:80] Number of filenames to read: 1 WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards. W0302 18:25:02.143152 25268 dataset_builder.py:86] num_readers has been reduced to 1 to match input file shards. WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\builders\dataset_builder.py
parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`. W0302 18:25:02.145147 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\builders\dataset_builder.py parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`. WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\builders\dataset_builder.py:235: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.data.Dataset.map() W0302 18:25:02.164129 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\object_detection-0.1-py3.10.egg\object_detection\builders\dataset_builder.py:235: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.data.Dataset.map() WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py
sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version. Instructions for updating: Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead. W0302 18:25:06.936405 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version. Instructions for updating: Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead. WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating: `seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead. W0302 18:25:09.047379 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version. Instructions for updating: `seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead. WARNING:tensorflow:From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. W0302 18:25:10.205773 25268 deprecation.py:350] From C:\Users\yasser\Desktop\website\2023\TensorFlow Object Detection\ThingsObjectDetection\odthings\lib\site-packages\tensorflow\python\util\dispatch.py to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. it stuck at the "Use tf.cast instead." and never move from thereHere is the cpu and gpu status CPUGPU
I tried many things from restating the kernel, give more time before end the train, reduce the batch size from 4 to 2, reduce step train from 4000 to 2000, increase labels(2 --> 3 type of picture {phone, cup, perfume}). still doesn't work
Источник: https://stackoverflow.com/questions/780 ... ver-starts
