8/6/2020 How to train an object detection model with mmdetection | DLology

How to train an object detection model with mmdetection

Home (/) / Blog (/blog/) / How to train an object detection model with mmdetection

Posted by: Chengwei (/blog/author/Chengwei/) 1 year, 1 month ago

(6 Comments (/blog/how-to-train-an-object-detection-model-with-mmdetection/#disqus_thread))

A while back you have learned how to train an object detection model with TensorFlow object detection API, and Google Colab's free GPU, if you haven't, check it out in the
post (https://www.dlology.com/blog/how-to-train-an-object-detection-model-easy-for-free/). The models in TensorFlow object detection are quite dated and missing updates for
the state of the art models like Cascade RCNN and RetinaNet. While there is a counterpart for Pytorch similar to that called mmdetection (https://github.com/open-
mmlab/mmdetection) which include more pre-trained state of the art object detection models for us to train custom data with, however setting it up requires a nontrivial amount
of time spent on installing the environment, setting up the config file, and dataset in the right format. The good news is you can skip those boring stuff and jump directly into
the fun part to train your model.

Here is an overview of how to make it happen,

1. Annotate some images, and make train/test split.

2. Run the Colab notebook (https://colab.research.google.com/github/Tony607/mmdetection_object_detection_demo/blob/master/mmdetection_train_custom_data.ipynb) to

train your model.

Step 1: Annotate some images and make train/test split

It is only necessary if you want to use your images instead of ones comes with my repository (https://github.com/Tony607/mmdetection_object_detection_demo). Start
by forking my repository (https://github.com/Tony607/mmdetection_object_detection_demo) and delete the data folder in the project directory so you can start fresh with your
custom data.

If you took your images by your phone, the image resolution might be 2K or 4K depends on your phone's setting. In that case, we will scale down the image for reduced
overall dataset size, and faster training speed.

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 1/7
8/6/2020 How to train an object detection model with mmdetection | DLology
You can use the resize_images.py (https://github.com/Tony607/mmdetection_object_detection_demo/blob/master/resize_images.py) script in the repository to resize your
images.

First, save all your photos to one folder outside of the project directory so they won't get accidentally uploaded to GitHub later. Ideally, all photo came with jpg extension.
Then run this script to resize all photos, and save them to the project directory.

python resize_images.py --raw-dir <photo_directory> --save-dir ./data/VOCdevkit/VOC2007/ImageSets --ext jpg --target-size "(800, 600)"

You might wonder why "VOC" in the path, that is because of the annotation tool we use generates Pascal VOC (http://host.robots.ox.ac.uk/pascal/VOC/) formatted annotation
XML files. It is not necessary to dig into the actual format of the XML file since the annotation tool handles all of that. You guessed it, that is the same tool we use previously
(https://www.dlology.com/blog/how-to-train-an-object-detection-model-easy-for-free/), LabelImg (https://tzutalin.github.io/labelImg/), works both on Windows and Linux.

Download LabelImg (https://tzutalin.github.io/labelImg/)and open it up,

1. Verify "PascalVOC" is selected, that is the default annotation format.

2. Open your resized image folder " ./data/VOCdevkit/VOC2007/ImageSets " for annotation.

3. Change save directory for the XML annotation files to " ./data/VOCdevkit/VOC2007/Annotations ".

As usual, use shortcuts ( w : draw box, d : next file, a : previous file, etc.) to accelerate the annotation.

Once it is done, you will find those XML files located in " ./data/VOCdevkit/VOC2007/Annotations " folder with the same file base names as your image files.

For the train/test split, you are going to create two files, each one containing a list of file base names, one name per line. Those two text files will be located in the folder
" data/VOC2007/ImageSets/Main " named trainval.txt and test.txt respectively. If you don't want to type all the file name by hand, try cd into the " Annotations " directory
and run the shell,

ls -1 | sed -e 's/\.xml$//' | sort -n

That will give you a list of nicely sorted file base names, just split them into two parts, and paste into those two text files.

Now you have the data directory structure similar to this one below.

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 2/7
8/6/2020 How to train an object detection model with mmdetection | DLology

data
└── VOC2007
├── Annotations
│ ├── 0.xml
│ ├── ...
│ └── 9.xml
├── ImageSets
│ └── Main
│ ├── test.txt
│ └── trainval.txt
└── JPEGImages
├── 0.jpg
├── ...
└── 9.jpg

Update your fork of the GitHub repository (https://github.com/Tony607/mmdetection_object_detection_demo) with your labeled datasets so you can clone it with Colab.

git add --al

git commit -m "Update datasets"
git push

Train the model on Colab Notebook

We are ready to launch the Colab notebook
(https://colab.research.google.com/github/Tony607/mmdetection_object_detection_demo/blob/master/mmdetection_train_custom_data.ipynb) and fire up the training. Similar
to TensorFlow object detection API, instead of training the model from scratch, we will do transfer learning from a pre-trained backbone such as resnet50 specified in the
model config file.

The notebook allows you to select the model config and set the number of training epochs.

Right now, I only tested with two model configs, faster_rcnn_r50_fpn_1x, and cascade_rcnn_r50_fpn_1x, while other configs can be incorporated as demonstrated in the
notebook.

The notebook handles several things before training the model,

1. Installing mmdetection and its dependencies.

2. Replacing "CLASSES" in voc.py file with your custom dataset class labels.
3. Modifying your selected model config file. Things like updating the number of classes to match with your dataset, changing dataset type to VOCDataset, setting the total
training epoch number and more.

After that, it will re-run the mmdetection package installing script so the changes to the voc.py file will be updated to the system python packages.

%cd {mmdetection_dir}
!python setup.py install

Since your data directory resides outside of the mmdetection directory, we have the following cell in the notebook which creates a symbolic link into the project data directory.

os.makedirs("data/VOCdevkit", exist_ok=True)
voc2007_dir = os.path.join(project_name, "data/VOC2007")
os.system("ln -s {} data/VOCdevkit".format(voc2007_dir))

Then start the training.

!python tools/train.py {config_fname}

The training time depends on the size of your datasets and number of training epochs, my demo takes several minutes to complete with Colab's Tesla T4 GPU.

After training, you can test drive the model with an image in the test set like so.

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 3/7
8/6/2020 How to train an object detection model with mmdetection | DLology

%cd {mmdetection_dir}
from mmcv.runner import load_checkpoint
from mmdet.apis import inference_detector, show_result, init_detector

checkpoint_file = os.path.join(mmdetection_dir, work_dir, "latest.pth")

score_thr = 0.8

# build the model from a config file and a checkpoint file

model = init_detector(config_fname, checkpoint_file)

# test a single image and show the results

img = 'data/VOCdevkit/VOC2007/JPEGImages/15.jpg'
result = inference_detector(model, img)
show_result(img, result, model.CLASSES, score_thr=score_thr, out_file="result.jpg")

# Show the image with bbox overlays.

from IPython.display import Image
Image(filename='result.jpg')

And here is the result as you expected,

Conclusion and further reading

This tutorial shows you how to train a Pytorch mmdetection object detection model with your custom dataset, and minimal effort on Google Colab Notebook.

If you are using my GitHub repo, you probably noticed that mmdetection is included as a submodule, to update that in the future run this command.

git submodule update --recursive

Considering training with another model config? You can find a list of config files here (https://github.com/open-mmlab/mmdetection/tree/master/configs) as well as their specs
(https://github.com/open-mmlab/mmdetection/blob/master/MODEL_ZOO.md#baselines) such as the complexity(Mem(GB)), and accuracy(box AP). Then start by adding the
config file to MODELS_CONFIG at the start of the notebook
(https://colab.research.google.com/github/Tony607/mmdetection_object_detection_demo/blob/master/mmdetection_train_custom_data.ipynb).

Resources you might find helpful,

mmdetection (https://github.com/open-mmlab/mmdetection) - GitHub repository.

LabelImg (https://tzutalin.github.io/labelImg/) - The Annotation tool used in this tutorial.
my repository (https://github.com/Tony607/mmdetection_object_detection_demo) for this tutorial.

In future posts, we will look into benchmarking those custom trained model as well as their deployment to edge computing devices, stay tuned and happy coding!

Tags: deep learning (/blog/tag/deep-learning/), tutorial (/blog/tag/tutorial/), pytorch (/blog/tag/pytorch/)

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 4/7
8/6/2020 How to train an object detection model with mmdetection | DLology
August (/blog/archive/2018/8/) (5)
July (/blog/archive/2018/7/) (4)
June (/blog/archive/2018/6/) (4)
May (/blog/archive/2018/5/) (4)
April (/blog/archive/2018/4/) (6)
March (/blog/archive/2018/3/) (5)
February (/blog/archive/2018/2/) (3)
January (/blog/archive/2018/1/) (4)
2017

December (/blog/archive/2017/12/) (4)

November (/blog/archive/2017/11/) (4)
October (/blog/archive/2017/10/) (3)
September (/blog/archive/2017/9/) (2)

Categories
deep learning (/blog/category/deep-learning/) (79)
edge computing (/blog/category/edge-computing/) (17)
Keras (/blog/category/keras/) (48)
NLP (/blog/category/nlp/) (8)
python (/blog/category/python/) (69)
PyTorch (/blog/category/pytorch/) (7)
tensorflow (/blog/category/tensorflow/) (35)

Tags
tutorial (/blog/tag/tutorial/) (56) Sentiment analysis (/blog/tag/sentiment-analysis/) (3) keras (/blog/tag/keras/) (35)
deep learning (/blog/tag/deep-learning/) (57) pytorch (/blog/tag/pytorch/) (2)
Authors
Chengwei (/blog/author/Chengwei/) (85)

Feeds
RSS (/blog/feeds/rss/) / Atom (/blog/feeds/atom/)

About Me (/about- Blog (/blog/) Privacy Policy Support (/support/)

me/) (/privacy-policy/)

 (https://twitter.com/TonyZhang607)  (https://www.facebook.com/chengwei.zhang.96)  (https://github.com/Tony607)  (https://www.linkedin.com/in/chengweizhang/)

Powered by Mezzanine (http://mezzanine.jupo.org) and Django (https://www.djangoproject.com) | Theme by Bootstrap (https://getbootstrap.com)

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 7/7
8/6/2020 How to train an object detection model with mmdetection | DLology

Current rating: 4.5 1 2 3 4 5 Rate

Share on Twitter (https://twitter.com/intent/tweet?url=https%3A//www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/&text=How%20to%20train%20an%20object%

Share on Facebook (https://www.facebook.com/sharer/sharer.php?u=https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/)

← How to do Transfer learning with Efficientnet (/blog/transfer-learning-with-efficientnet/)

How to create custom COCO data set for object detection → (/blog/how-to-create-custom-coco-data-set-for-object-detection/)

Related posts
How to create custom COCO data set for instance segmentation (/blog/how-to-create-custom-coco-data-set-for-instance-segmentation/)

Comments
ALSO ON DLOLOGY

How to run Keras Multi-class How to convert trained How to run

model on Jetson … classification with … Keras model to a … Object Det
a year ago • 2 comments 2 years ago • 4 comments 2 years ago • 9 comments a year ago • 1

This tutorial shows the This tutorial will show you You are going to learn step (Comments)
complete process to get a how to apply focal loss to by step how to freeze and you have lea
Keras model running on … train a multi-class … convert your trained … a Keras ima

What do you think?

8 Responses

Upvote Funny Love Surprised Angry Sad

6 Comments dlology 🔒 Disqus' Privacy Policy 

1 Login

 Recommend 1 t Tweet f Share Sort by Best

Join the discussion…

LOG IN WITH
OR SIGN UP WITH DISQUS ?

Name

Mukul Vishwas • 2 months ago

I am trying to run the colab and while training i am getting this error:
I am not sure why???????

2020-05-23 10:36:04,458 - mmdet - WARNING - The model and loaded state dict do not match
exactly

unexpected key in source state_dict: fc.weight, fc.bias

Traceback (most recent call last):

File "/content/mmdetection_object_detection_demo/mmdetection/tools/train.py", line 161, in
<module>
main()
File "/content/mmdetection_object_detection_demo/mmdetection/tools/train.py", line 136, in main
datasets = [build_dataset(cfg.data.train)]
File "/usr/local/lib/python3.6/dist-packages/mmdet-2.0.0+0367dd9-py3.6-linux-
x86_64.egg/mmdet/datasets/builder.py", line 56, in build_dataset
build_dataset(cfg['dataset'], default_args), cfg['times'])
File "/usr/local/lib/python3.6/dist-packages/mmdet-2.0.0+0367dd9-py3.6-linux-
x86 64 egg/mmdet/datasets/builder py" line 61 in build dataset
see more

1△ ▽ • Reply • Share ›
https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 5/7
8/6/2020 How to train an object detection model with mmdetection | DLology
1 Reply Share ›

Marco Maiocchi • a year ago

In order to solve the deform_conv_cuda error i ran the following in a previous cell

!python setup.py develop

1△ ▽ • Reply • Share ›

Surya Rasp > Marco Maiocchi • 5 months ago

thanks for sharing the tip to solve the issue
△ ▽ • Reply • Share ›

Enrque Apolo • 4 months ago

Great tutorial! I have a problem. I'd like to store all in my Google Drive. In other words, How can
execute everything inside drive/My Drive, because when I tried some errors ocurr.
△ ▽ • Reply • Share ›

Ninad Thorat • a year ago

ImportError: cannot import name 'deform_conv_cuda'
. In Test-predict block.
△ ▽ • Reply • Share ›

veeragoni ashish > Ninad Thorat • a year ago

PyTorch version 1.1 mmdetection should be installed with "python setup.py develop or pip
install -v -e ." ! !
PyTorch-0.4.1 version mmdetection is compiled with "./compile.sh" and installed with
python setup.py install!
△ ▽ • Reply • Share ›

✉ Subscribe d Add Disqus to your siteAdd DisqusAdd ⚠ Do Not Sell My Data

Recent Posts
Accelerated Deep Learning inference from your browser (/blog/accelerated-deep-learning-inference-from-your-browser/)
How to run SSD Mobilenet V2 object detection on Jetson Nano at 20+ FPS (/blog/how-to-run-ssd-mobilenet-v2-object-detection-on-jetson-nano-at-20-fps/)
Automatic Defect Inspection with End-to-End Deep Learning (/blog/automatic-defect-inspection-with-end-to-end-deep-learning/)
How to train Detectron2 with Custom COCO Datasets (/blog/how-to-train-detectron2-with-custom-coco-datasets/)
Getting started with VS CODE remote development (/blog/getting-started-with-vscode-remote-development/)

Archive
2020

June (/blog/archive/2020/6/) (1)

2019

December (/blog/archive/2019/12/) (1)

November (/blog/archive/2019/11/) (1)
October (/blog/archive/2019/10/) (1)
September (/blog/archive/2019/9/) (3)
August (/blog/archive/2019/8/) (1)
July (/blog/archive/2019/7/) (2)
June (/blog/archive/2019/6/) (2)
May (/blog/archive/2019/5/) (3)
April (/blog/archive/2019/4/) (3)
March (/blog/archive/2019/3/) (1)
February (/blog/archive/2019/2/) (2)
January (/blog/archive/2019/1/) (2)
2018

December (/blog/archive/2018/12/) (3)

November (/blog/archive/2018/11/) (3)
October (/blog/archive/2018/10/) (3)
September (/blog/archive/2018/9/) (5)

https://www.dlology.com/blog/how-to-train-an-object-detection-model-with-mmdetection/ 6/7