From d221431b0114130189fb7234675ed701536b59f7 Mon Sep 17 00:00:00 2001 From: Konrad <54865575+KonradDanielewski@users.noreply.github.com> Date: Tue, 13 Jun 2023 11:26:55 +0200 Subject: [PATCH 1/7] Update _toc.yml --- _toc.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/_toc.yml b/_toc.yml index 2dca52fb7..e674215a4 100644 --- a/_toc.yml +++ b/_toc.yml @@ -45,6 +45,7 @@ parts: - file: docs/recipes/DLCMethods - file: docs/recipes/OpenVINO - file: docs/recipes/flip_and_rotate + - file: docs/recipes/pose_cfg_file_breakdown - caption: Mission & Contribute chapters: - file: docs/MISSION_AND_VALUES From e3cc800c82601dc77a1e6339046a34300d1a835e Mon Sep 17 00:00:00 2001 From: Konrad <54865575+KonradDanielewski@users.noreply.github.com> Date: Tue, 13 Jun 2023 11:27:28 +0200 Subject: [PATCH 2/7] add pose_cfg_recipe --- docs/recipes/pose_cfg_file_breakdown.md | 213 ++++++++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 docs/recipes/pose_cfg_file_breakdown.md diff --git a/docs/recipes/pose_cfg_file_breakdown.md b/docs/recipes/pose_cfg_file_breakdown.md new file mode 100644 index 000000000..8810e8567 --- /dev/null +++ b/docs/recipes/pose_cfg_file_breakdown.md @@ -0,0 +1,213 @@ +# The `pose_cfg.yaml` Handbook +Hello! Mabuhay! Hola! +In this notebook, we will have a rundown on the following pose config parameters related to models' training and data augmentation: + +# 2. What is *pose_cfg.yml*? + +The `pose_cfg.yaml` file offers easy access to a range of training parameters that the user may want or have to adjust depending on the used dataset and task. This recipe is aimed at giving an average user an intuition on those hyperparameters and situations in which adressing them can be useful + +# 3. Full parameter list + +- [The `pose_cfg.yaml` Handbook](#the-pose_cfgyaml-handbook) +- [2. What is *pose\_cfg.yml*?](#2-what-is-pose_cfgyml) +- [3. Full parameter list](#3-full-parameter-list) + - [3.1 Training Hyperparameters](#31-training-hyperparameters) + - [3.1.A `max_input_size` and `min_input_size`](#31a-max_input_size-and-min_input_size) + - [3.1.B `global_scale`](#31b-global_scale) + - [3.1.C `batch_size`](#31c-batch_size) + - [3.1.D `pos_dist_thresh`](#31d-pos_dist_thresh) + - [3.1.E `pafwidth`](#31e-pafwidth) + - [3.2 Data augmentation parameters](#32-data-augmentation-parameters) + - [Geometric transformations](#geometric-transformations) + - [3.2.1 `scale_jitter_lo` and `scale_jitter_up`](#321-scale_jitter_lo-and-scale_jitter_up) + - [3.1.2 `rotation`](#312-rotation) + - [3.2.3 `rotratio` (rotation ratio)](#323-rotratio-rotation-ratio) + - [3.2.4 `fliplr` (or a horizontal flip)](#324-fliplr-or-a-horizontal-flip) + - [3.2.5 `crop_size`](#325-crop_size) + - [3.2.6 `crop_ratio`](#326-crop_ratio) + - [3.2.7 `max_shift`](#327-max_shift) + - [3.2.8 `crop_sampling`](#328-crop_sampling) + - [Kernel transformations](#kernel-transformations) + - [3.2.9 `sharpening` and `sharpenratio`](#329-sharpening-and-sharpenratio) + - [3.2.10 `edge`](#3210-edge) +- [References](#references) + + +## 3.1 Training Hyperparameters + + +### 3.1.A `max_input_size` and `min_input_size` +The default values are `1500` and `64` respectively. + +💡Pro-tip:💡 +- change `max_input_size` when resolution of the video is higher than 1500x1500 or when `scale_jitter_up` will possibly go over that value +- change `min_input_size` when resolution of the video is smaller than 64x64 or when `scale_jitter_lo` will possibly go below that value + + +### 3.1.B `global_scale` +The default value is `0.8`. It's the most basic, first scaling that happens to all images in the training queue. + +💡Pro-tip:💡 +- With images that are low resolution or lack detail, it may be beneficial to increase the `global_scale` to 1, to keep original size and retain as much information as possible. + +### 3.1.C `batch_size` + + +The default for single animal projects is 1 and for maDLC projects it's `8`. It's the number of frames used per training iteration. + +In both cases you can increase the batchsize up to the limit of your GPU memory and train for a lower number of iterations. The relationship between number of iterations and `batch_size` is not linear so `batch_size: 8` doesn't mean you can train for 8x less iterations, but like with every training, plateauing loss can be treated as an indicator of reaching optimal performance. + +💡Pro-tip:💡 +- Having a higher `batch_size` can be beneficial in terms of models' generalization + +___________________________________________________________________________________ + +Values mentioned above and the augmenatation parameters are often intuitive and knowing our own data we are able to decide on what will and won't be beneficial. Unfortunately, not all hyperparameters are this simple or intuitive. Two of the parameters that might require some tuning on challanging datasets are `pafwidth` and `pos_dist_thresh`. + + +### 3.1.D `pos_dist_thresh` +The default value is `17`. It's the size of a window withing which detections are considered positive training samples, meaning they tell the model, that it's going in the right direction. + + +### 3.1.E `pafwidth` +The default value is `20`. PAF stands for part affinity fields. It is a method of learning associations between pairs of bodyparts by preserving location and orientation of the limb (connection between two keypoints). This learned part affinity helps in proper animal assembly, making the model less prone to associating bodyparts of one individual with those of another. [1](#ref1) + + +## 3.2 Data augmentation parameters +In the simplest form, we can think of data augmentation as something similar to imagination or dreaming. Humans imagine diferent scenarios based on experience, ultimately allowing us to gain a better understanding of our world. [2, 3, 4](#references) + +In a similar way, we train our models to different types of "imagined" scenarios, which we limit to the foreseeable ones, so we ultimately get a robust model that can more likely handle new data and scenes. + +Classes of data augmentations, characterized by their nature, are given by: +- [**Geometric transformations**](#geometric) + 1. [`scale_jitter_lo` and `scale_jitter_up`](#scale_jitter) + 2. [`rotation`](#rot) + 3. [`rotratio`](#rotratio) + 4. [`mirror`](#mirror) + 5. [`crop size`](#crop_size) + 6. [`crop ratio`](#crop_ratio) + 7. [`max shift`](#max_shift) + 8. [`crop sampling`](#crop_sampling) +- [**Kernel transformations**](#kernel) + 9. [`sharpening` and `sharpen_ratio`](#sharp) + 10. [`edge_enhancement`](#edge) + + +### Geometric transformations +**Geometric transformations** such as *flipping*, *rotating*, *translating*, *cropping*, *scaling*, and *injecting noise*, which are very good for positional biases present in the training data. + + +### 3.2.1 `scale_jitter_lo` and `scale_jitter_up` +*Scale jittering* resizes an image within a given resize range. This allows the model to learn from different sizes of objects in the scene, therefore increasing its robustness to generalize especially on newer scenes or object sizes. + +The image below, retrieved from [3](#ref3), illustrates the difference between two scale jittering methods. + +![scale_jittering.png](attachment:scale_jittering.png) + +During training, each image is randomly scaled within the range `[scale_jitter_lo, scale_jitter_up]` to augment training data. The default values for these two parameters are: +- `scale_jitter_lo = 0.5` +- `scale_jitter_up = 1.25` + +💡Pro-tips:💡 +- ⭐⭐⭐ If the target animal/s do not have an incredibly high variance in size throughout the video (e.g. jumping or moving towards the static camera), keeping the **default** values **unchanged** will give just enough variability in the data for the model to generalize better ✅ + +- ⭐⭐However, you may want to adjust these parameters if you want your model to: + - handle new data with possibly **larger (25% bigger than original)** animal subjects ➡️ in this scenario, increase the value of *scale_jitter_up* + - handle new data with possibly **smaller (50% smaller than the original)** animal subjects ➡️ in this scenario, decrease the value of *scale_jitter_lo* + - **generalize well in new set-ups/environments** with minimal to no pre-training + ⚠️ But as a consequence, **training time will take longer**.😔🕒 +- ⭐If you have a fully static camera set-up and the sizes of the animals do not vary much, you may also try to **shorten** this range to **reduce training time**.😃🕒(⚠️ but, as a consequence, your model might only fit your data and not generalize well) + + +### 3.1.2 `rotation` +*Rotation augmentations* are done by rotating the image right or left on an axis between $1^{\circ}$ and $359^{\circ}$. Te safety of rotation augmentations is heavily determined by the rotation degree parameter. Slight rotations such as between $+1^{\circ}$ and $+20^{\circ}$ or $-1^{\circ}$ to $-20^{\circ}$ is generally an acceptable range. Keep in mind that as the rotation degree increases, the precision of the label placement can decrease + +The image below retreived from [2](#ref2) illustrates the difference between the different rotation degrees. +![augset_rot.png](attachment:augset_rot.png) + +During training, each image is rotated $+/-$ the `rotation` degree parameter set. By default, this parameter is set to `25`, which means that the images are augmented with a $+25^{\circ}$ rotation of itself and a $-25^{\circ}$ degree rotation of itself. Should you want to opt out of this augmentation, set the rotation value to `False`. + +💡Pro-tips:💡 +- ⭐If you have labelled all the possible rotations of your animal/s, keeping the **default** value **unchanged** is **enough** ✅ + +- However, you may want to adjust this parameter if you want your model to: + - handle new data with new rotations of the animal subjects + - handle the possibly unlabelled rotations of your minimally-labelled data + - But as a consequence, the more you increase the rotation degree, the more the original keypoint labels may not be preserved + + +### 3.2.3 `rotratio` (rotation ratio) +This parameter in the DLC module is given by the percentage of sampled data to be augmented from your training data. The default value is set to `0.4` or $40\%$. This means that there is a $40\%$ chance that images within the current batch will be rotated. + +💡Pro-tip:💡 +- ⭐ Generally, keeping the **default** value **unchanged** is **enough** ✅ + + +### 3.2.4 `fliplr` (or a horizontal flip) +**Mirroring** or otherwise called **horizontal axis fipping** is much more common than fipping the vertical axis. This augmentation is one of the easiest to implement and has proven useful on datasets such as CIFAR-10 and ImageNet. However, on datasets involving text recognition such as MNIST or SVHN, this is not a label-preserving transformation. + +The image below is an illustration of this property (shown on the right most column). +![augset_flip.png](attachment:augset_flip.png) + +This parameter randomly flips an image horizontally to augment training data. +By default, this parameter is set to `False` especially on poses with mirror symmetric joints (for example, so the left hand and right hand are not swapped). + +💡Pro-tip:💡 +- ⭐ If you work with labels with symmetric joints, keep the **default** value **unchanged** - unless the dataset is biased (animal moves mostly in one direction, but sometimes in the opposite)✅ +- Keeping the default value to `False` will work well in most cases. + + +### 3.2.5 `crop_size` +Cropping consists of removing unwanted pixels from the image, thus selecting a part of the image and discarding the rest, reducing the size of the input. + +In DeepLabCut *pose_config.yaml* file, by default `crop_size` is set to (`400,400`), width and height respectively. This means it will cut out parts of an image of this size. + +💡Pro-tip:💡 + - If your images are very large, you could consider increasing the crop size. However, be aware that you'll need a strong GPU or you will hit memory errors! + - If your images are very small, you could consider decreasing the crop size. + + +### 3.2.6 `crop_ratio` + Also, the number of frames to be cropped is defined by the variable `cropratio`, which by default is set to `0.4`. That means that there is a $40\%$ the images within current batch will be cropped. By default this value works well. + + +### 3.2.7 `max_shift` + + The crop shift between each cropped image is defined by `max_shift` variable, which explains the max relative shift to the position of the crop centre. By default is set to `0.4`, which means it will be displaced 40% max from the center to not apply identical cropping each time the same image is encountered during training - this is especially important for `density` and `hybrid` cropping methods. + +The image below is modified from +[2](#references). +![cropping.png](attachment:cropping.png) + + +### 3.2.8 `crop_sampling` +Likewise, there are different cropping sampling methods (`crop_sampling`), we can use depending on how our image looks like. + +💡Pro-tips💡 +- For highly crowded scenes `hybrid` and `density` approaches will work best. +- `uniform` will take out random parts of the image, disregarding the annotations completely +- 'keypoint' centers on a random keypoint and crops based on that location - might be best in preserving whole animal (if reasonable `crop_size` is used) + + +### Kernel transformations +Kernel filters are very popular in image processing to sharpen and blur images. Intuitively, blurring an image might increase the motion blur resistance during testing. Otherwise, sharpening for data enhancement could result in capturing more detail on objects of interest. + + +### 3.2.9 `sharpening` and `sharpenratio` +In DeepLabCut *pose_config.yaml* file, by default `sharpening` is set to `False`, but if we want to use this type of data augmentation, we can set it `True` and specify a value for `sharpenratio`, which by default is set to `0.3`. Blurring is not defined in the *pose_config.yaml*, but if the user finds it convenient, it can be added to the data augmentation pipeline. + +The image below is modified from +[2](#references). +![kernelfilter.png](attachment:kernelfilter.png) + + +### 3.2.10 `edge` +Concerning sharpeness, we have an additional parameter, `edge` enhancement, which enhances edge contrast of an image to improve its apparent sharpness. Likewise, by default this parameter is set `False`, but if you want to include it you just need to set it `True`. + +# References +
    +
  1. Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2017). Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7291-7299).https://openaccess.thecvf.com/content_cvpr_2017/html/Cao_Realtime_Multi-Person_2D_CVPR_2017_paper.html
    2. +
  2. Mathis, A., Schneider, S., Lauer, J., & Mathis, M. W. (2020). A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives. In Neuron (Vol. 108, Issue 1, pp. 44-65). Elsevier BV. https://doi.org/10.1016/j.neuron.2020.09.017
    3. +
  3. Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.-Y., Cubuk, E. D., Le, Q. V., & Zoph, B. (2020). Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation (Version 2). arXiv. https://doi.org/10.48550/ARXIV.2012.07177
    4. +
  4. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on Image Data Augmentation for Deep Learning. In Journal of Big Data (Vol. 6, Issue 1). Springer Science and Business Media LLC. https://doi.org/10.1186/s40537-019-0197-0
    5. +
From ab9e8a2e2c263e8b65687719d41b784b24a65ade Mon Sep 17 00:00:00 2001 From: rizarae-p Date: Tue, 13 Jun 2023 13:22:08 +0200 Subject: [PATCH 3/7] Corrected grammatical errors in the document --- docs/recipes/pose_cfg_file_breakdown.md | 54 ++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/recipes/pose_cfg_file_breakdown.md b/docs/recipes/pose_cfg_file_breakdown.md index 8810e8567..2752fc16b 100644 --- a/docs/recipes/pose_cfg_file_breakdown.md +++ b/docs/recipes/pose_cfg_file_breakdown.md @@ -4,7 +4,7 @@ In this notebook, we will have a rundown on the following pose config parameters # 2. What is *pose_cfg.yml*? -The `pose_cfg.yaml` file offers easy access to a range of training parameters that the user may want or have to adjust depending on the used dataset and task. This recipe is aimed at giving an average user an intuition on those hyperparameters and situations in which adressing them can be useful +The `pose_cfg.yaml` file offers easy access to a range of training parameters that the user may want or have to adjust depending on the used dataset and task. This recipe is aimed at giving an average user an intuition on those hyperparameters and situations in which addressing them can be useful. # 3. Full parameter list @@ -37,46 +37,46 @@ The `pose_cfg.yaml` file offers easy access to a range of training parameters th ### 3.1.A `max_input_size` and `min_input_size` -The default values are `1500` and `64` respectively. +The default values are `1500` and `64`, respectively. 💡Pro-tip:💡 -- change `max_input_size` when resolution of the video is higher than 1500x1500 or when `scale_jitter_up` will possibly go over that value -- change `min_input_size` when resolution of the video is smaller than 64x64 or when `scale_jitter_lo` will possibly go below that value +- change `max_input_size` when the resolution of the video is higher than 1500x1500 or when `scale_jitter_up` will possibly go over that value +- change `min_input_size` when the resolution of the video is smaller than 64x64 or when `scale_jitter_lo` will possibly go below that value ### 3.1.B `global_scale` The default value is `0.8`. It's the most basic, first scaling that happens to all images in the training queue. 💡Pro-tip:💡 -- With images that are low resolution or lack detail, it may be beneficial to increase the `global_scale` to 1, to keep original size and retain as much information as possible. +- With images that are low resolution or lack detail, it may be beneficial to increase the `global_scale` to 1, to keep the original size and retain as much information as possible. ### 3.1.C `batch_size` -The default for single animal projects is 1 and for maDLC projects it's `8`. It's the number of frames used per training iteration. +The default for single animal projects is 1, and for maDLC projects it's `8`. It's the number of frames used per training iteration. -In both cases you can increase the batchsize up to the limit of your GPU memory and train for a lower number of iterations. The relationship between number of iterations and `batch_size` is not linear so `batch_size: 8` doesn't mean you can train for 8x less iterations, but like with every training, plateauing loss can be treated as an indicator of reaching optimal performance. +In both cases, you can increase the batchsize up to the limit of your GPU memory and train for a lower number of iterations. The relationship between the number of iterations and `batch_size` is not linear so `batch_size: 8` doesn't mean you can train for 8x less iterations, but like with every training, plateauing loss can be treated as an indicator of reaching optimal performance. 💡Pro-tip:💡 - Having a higher `batch_size` can be beneficial in terms of models' generalization ___________________________________________________________________________________ -Values mentioned above and the augmenatation parameters are often intuitive and knowing our own data we are able to decide on what will and won't be beneficial. Unfortunately, not all hyperparameters are this simple or intuitive. Two of the parameters that might require some tuning on challanging datasets are `pafwidth` and `pos_dist_thresh`. +Values mentioned above and the augmentation parameters are often intuitive, and knowing our own data, we are able to decide on what will and won't be beneficial. Unfortunately, not all hyperparameters are this simple or intuitive. Two parameters that might require some tuning on challenging datasets are `pafwidth` and `pos_dist_thresh`. ### 3.1.D `pos_dist_thresh` -The default value is `17`. It's the size of a window withing which detections are considered positive training samples, meaning they tell the model, that it's going in the right direction. +The default value is `17`. It's the size of a window within which detections are considered positive training samples, meaning they tell the model that it's going in the right direction. ### 3.1.E `pafwidth` -The default value is `20`. PAF stands for part affinity fields. It is a method of learning associations between pairs of bodyparts by preserving location and orientation of the limb (connection between two keypoints). This learned part affinity helps in proper animal assembly, making the model less prone to associating bodyparts of one individual with those of another. [1](#ref1) +The default value is `20`. PAF stands for part affinity fields. It is a method of learning associations between pairs of bodyparts by preserving the location and orientation of the limb (the connection between two keypoints). This learned part affinity helps in proper animal assembly, making the model less prone to associating bodyparts of one individual with those of another. [1](#ref1) ## 3.2 Data augmentation parameters -In the simplest form, we can think of data augmentation as something similar to imagination or dreaming. Humans imagine diferent scenarios based on experience, ultimately allowing us to gain a better understanding of our world. [2, 3, 4](#references) +In the simplest form, we can think of data augmentation as something similar to imagination or dreaming. Humans imagine different scenarios based on experience, ultimately allowing us to better understand our world. [2, 3, 4](#references) -In a similar way, we train our models to different types of "imagined" scenarios, which we limit to the foreseeable ones, so we ultimately get a robust model that can more likely handle new data and scenes. +Similarly, we train our models to different types of "imagined" scenarios, which we limit to the foreseeable ones, so we ultimately get a robust model that can more likely handle new data and scenes. Classes of data augmentations, characterized by their nature, are given by: - [**Geometric transformations**](#geometric) @@ -98,7 +98,7 @@ Classes of data augmentations, characterized by their nature, are given by: ### 3.2.1 `scale_jitter_lo` and `scale_jitter_up` -*Scale jittering* resizes an image within a given resize range. This allows the model to learn from different sizes of objects in the scene, therefore increasing its robustness to generalize especially on newer scenes or object sizes. +*Scale jittering* resizes an image within a given resize range. This allows the model to learn from different sizes of objects in the scene, therefore increasing its robustness to generalize, especially on newer scenes or object sizes. The image below, retrieved from [3](#ref3), illustrates the difference between two scale jittering methods. @@ -109,7 +109,7 @@ During training, each image is randomly scaled within the range `[scale_jitter_l - `scale_jitter_up = 1.25` 💡Pro-tips:💡 -- ⭐⭐⭐ If the target animal/s do not have an incredibly high variance in size throughout the video (e.g. jumping or moving towards the static camera), keeping the **default** values **unchanged** will give just enough variability in the data for the model to generalize better ✅ +- ⭐⭐⭐ If the target animal/s do not have an incredibly high variance in size throughout the video (e.g., jumping or moving towards the static camera), keeping the **default** values **unchanged** will give just enough variability in the data for the model to generalize better ✅ - ⭐⭐However, you may want to adjust these parameters if you want your model to: - handle new data with possibly **larger (25% bigger than original)** animal subjects ➡️ in this scenario, increase the value of *scale_jitter_up* @@ -120,9 +120,9 @@ During training, each image is randomly scaled within the range `[scale_jitter_l ### 3.1.2 `rotation` -*Rotation augmentations* are done by rotating the image right or left on an axis between $1^{\circ}$ and $359^{\circ}$. Te safety of rotation augmentations is heavily determined by the rotation degree parameter. Slight rotations such as between $+1^{\circ}$ and $+20^{\circ}$ or $-1^{\circ}$ to $-20^{\circ}$ is generally an acceptable range. Keep in mind that as the rotation degree increases, the precision of the label placement can decrease +*Rotation augmentations* are done by rotating the image right or left on an axis between $1^{\circ}$ and $359^{\circ}$. The safety of rotation augmentations is heavily determined by the rotation degree parameter. Slight rotations such as between $+1^{\circ}$ and $+20^{\circ}$ or $-1^{\circ}$ to $-20^{\circ}$ is generally an acceptable range. Keep in mind that as the rotation degree increases, the precision of the label placement can decrease -The image below retreived from [2](#ref2) illustrates the difference between the different rotation degrees. +The image below, retrieved from [2](#ref2), illustrates the difference between the different rotation degrees. ![augset_rot.png](attachment:augset_rot.png) During training, each image is rotated $+/-$ the `rotation` degree parameter set. By default, this parameter is set to `25`, which means that the images are augmented with a $+25^{\circ}$ rotation of itself and a $-25^{\circ}$ degree rotation of itself. Should you want to opt out of this augmentation, set the rotation value to `False`. @@ -132,7 +132,7 @@ During training, each image is rotated $+/-$ the `rotation` degree parameter set - However, you may want to adjust this parameter if you want your model to: - handle new data with new rotations of the animal subjects - - handle the possibly unlabelled rotations of your minimally-labelled data + - handle the possibly unlabelled rotations of your minimally-labeled data - But as a consequence, the more you increase the rotation degree, the more the original keypoint labels may not be preserved @@ -144,13 +144,13 @@ This parameter in the DLC module is given by the percentage of sampled data to b ### 3.2.4 `fliplr` (or a horizontal flip) -**Mirroring** or otherwise called **horizontal axis fipping** is much more common than fipping the vertical axis. This augmentation is one of the easiest to implement and has proven useful on datasets such as CIFAR-10 and ImageNet. However, on datasets involving text recognition such as MNIST or SVHN, this is not a label-preserving transformation. +**Mirroring**, otherwise called **horizontal axis fipping**, is much more common than flipping the vertical axis. This augmentation is one of the easiest to implement and has proven useful on datasets such as CIFAR-10 and ImageNet. However, on datasets involving text recognition, such as MNIST or SVHN, this is not a label-preserving transformation. -The image below is an illustration of this property (shown on the right most column). +The image below is an illustration of this property (shown on the right-most column). ![augset_flip.png](attachment:augset_flip.png) This parameter randomly flips an image horizontally to augment training data. -By default, this parameter is set to `False` especially on poses with mirror symmetric joints (for example, so the left hand and right hand are not swapped). +By default, this parameter is set to `False` especially on poses with mirror symmetric joints (for example, so the left hand and right hand are not swapped). 💡Pro-tip:💡 - ⭐ If you work with labels with symmetric joints, keep the **default** value **unchanged** - unless the dataset is biased (animal moves mostly in one direction, but sometimes in the opposite)✅ @@ -160,15 +160,15 @@ By default, this parameter is set to `False` especially on poses with mirror sym ### 3.2.5 `crop_size` Cropping consists of removing unwanted pixels from the image, thus selecting a part of the image and discarding the rest, reducing the size of the input. -In DeepLabCut *pose_config.yaml* file, by default `crop_size` is set to (`400,400`), width and height respectively. This means it will cut out parts of an image of this size. +In DeepLabCut *pose_config.yaml* file, by default, `crop_size` is set to (`400,400`), width, and height, respectively. This means it will cut out parts of an image of this size. 💡Pro-tip:💡 - - If your images are very large, you could consider increasing the crop size. However, be aware that you'll need a strong GPU or you will hit memory errors! + - If your images are very large, you could consider increasing the crop size. However, be aware that you'll need a strong GPU, or you will hit memory errors! - If your images are very small, you could consider decreasing the crop size. ### 3.2.6 `crop_ratio` - Also, the number of frames to be cropped is defined by the variable `cropratio`, which by default is set to `0.4`. That means that there is a $40\%$ the images within current batch will be cropped. By default this value works well. + Also, the number of frames to be cropped is defined by the variable `cropratio`, which is set to `0.4` by default. That means that there is a $40\%$ the images within the current batch will be cropped. By default, this value works well. ### 3.2.7 `max_shift` @@ -184,9 +184,9 @@ The image below is modified from Likewise, there are different cropping sampling methods (`crop_sampling`), we can use depending on how our image looks like. 💡Pro-tips💡 -- For highly crowded scenes `hybrid` and `density` approaches will work best. +- For highly crowded scenes, `hybrid` and `density` approaches will work best. - `uniform` will take out random parts of the image, disregarding the annotations completely -- 'keypoint' centers on a random keypoint and crops based on that location - might be best in preserving whole animal (if reasonable `crop_size` is used) +- 'keypoint' centers on a random keypoint and crops based on that location - might be best in preserving the whole animal (if reasonable `crop_size` is used) ### Kernel transformations @@ -194,7 +194,7 @@ Kernel filters are very popular in image processing to sharpen and blur images. ### 3.2.9 `sharpening` and `sharpenratio` -In DeepLabCut *pose_config.yaml* file, by default `sharpening` is set to `False`, but if we want to use this type of data augmentation, we can set it `True` and specify a value for `sharpenratio`, which by default is set to `0.3`. Blurring is not defined in the *pose_config.yaml*, but if the user finds it convenient, it can be added to the data augmentation pipeline. +In DeepLabCut *pose_config.yaml* file, by default, `sharpening` is set to `False`, but if we want to use this type of data augmentation, we can set it `True` and specify a value for `sharpenratio`, which by default is set to `0.3`. Blurring is not defined in the *pose_config.yaml*, but if the user finds it convenient, it can be added to the data augmentation pipeline. The image below is modified from [2](#references). @@ -202,7 +202,7 @@ The image below is modified from ### 3.2.10 `edge` -Concerning sharpeness, we have an additional parameter, `edge` enhancement, which enhances edge contrast of an image to improve its apparent sharpness. Likewise, by default this parameter is set `False`, but if you want to include it you just need to set it `True`. +Concerning sharpness, we have an additional parameter, `edge` enhancement, which enhances the edge contrast of an image to improve its apparent sharpness. Likewise, by default, this parameter is set `False`, but if you want to include it, you just need to set it `True`. # References
    From 7f4f0120424f825fc1f641bfb48eaf6e26788a64 Mon Sep 17 00:00:00 2001 From: rizarae-p Date: Tue, 13 Jun 2023 17:57:25 +0200 Subject: [PATCH 4/7] new cookbook on how to publish notebooks into the main DLC recipe book --- ...ng_notebooks_into_the_DLC_main_cookbook.md | 105 ++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md diff --git a/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md new file mode 100644 index 000000000..33043a1de --- /dev/null +++ b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md @@ -0,0 +1,105 @@ +# Publishing Notebooks into the Main DLC Cookbook +*Date: 13 June 2023* + + +## Introduction +Publishing notebooks into the main DLC cookbook can be done in a few easy steps! + +## Requirements +To accomplish this, you need the following installed: +- jupyter-book +- numpydoc +- nbconvert +- jupyter_contrib_nbextensions + + You can do this by running the following command: + *** + ``` + pip install jupyter-book numpydoc nbconvert jupyter_contrib_nbextensions + jupyter contrib nbextension install --user + ``` + *** +**Relevant Git repos:** +- DeepLabCut: https://github.com/DeepLabCut/DeepLabCut + ``` + git clone https://github.com/DeepLabCut/DeepLabCut.git + ``` +- DeepLabCut2023version (forked from main DLC repo): https://github.com/DeepLabCutAIResidency/DeepLabCut2023version + ``` + git clone https://github.com/DeepLabCutAIResidency/DeepLabCut2023version.git + ``` + +## Steps +1. Double check for spelling and grammatical errors (on [Grammarly](#https://grammarly.com/) or by using Jupyter notebook's spellcheck extension called `spellchecker`). + *** + ``` + jupyter nbextension enable spellchecker/main + ``` + *** + Once installed, restart your notebook, and when you load your notebook again, you will see the incorrectly spelled words highlighted in red. See example below: + +2. Convert your notebook into a Markdown file (.md). + *** + ``` + jupyter nbconvert --to markdown [notebook.ipynb] + ``` + *** + + +3. Move your newly converted Markdown file (.md) to directory: `docs/recipes/` + *** + ``` + cp file.md path+to+the+local+copy+of+DLC/DeepLabCut2023version/docs/recipes + ``` + *** +4. Add the path to your new_recipe.md under the Tutorials & Cookbook paths in the `DeepLabCut2023version/_toc.yml` file. + *** + ``` + - file: docs/recipes/pose_cfg_file_breakdown + ``` + *** + +5. Build your notebook into the DLC recipe book + *** + ``` + jupyter book build /absolute/path/to/the/DLC/repoDeepLabCut2023version + ``` + *** + Example: + *** + ``` + jupyter book build /Users/rae/DeepLabCut2023version + ``` + *** + The build log should look like below: +
    + +5. Test locally by checking the `index.html` file in `/Users/rae/Desktop/DeepLabCut2023version/_build/html/` + + +6. When everything is a-okay, commit to Git. If not, edit your file and go to back to step 1. + + **`git status`** to check the local changes in your current project + ``` + git status + ``` + **`git add`** to add your file/s to the commit bin + ``` + git add [filename] + ``` + + **`git commit`** to add your file/s to the commit bin + ``` + git commit -m "commit message here; make it descriptive!" + ``` + **`git pull`** or **`git rebase`** to update your local copy from the main branch. + ``` + git rebase + ``` + **`git push`** to push your changes to the main branch. + ``` + git push + ``` +5. When everything's clear, confirm your pull request on the Git website: https://github.com/DeepLabCutAIResidency/DeepLabCut2023version + +## All done! From 72758f46f6f76371d66c25ec8e902a8005322560 Mon Sep 17 00:00:00 2001 From: rizarae-p Date: Tue, 13 Jun 2023 18:04:00 +0200 Subject: [PATCH 5/7] updated link generator file to accommodate my new notebook in the cookbook --- _toc.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/_toc.yml b/_toc.yml index e674215a4..bd721320a 100644 --- a/_toc.yml +++ b/_toc.yml @@ -46,6 +46,7 @@ parts: - file: docs/recipes/OpenVINO - file: docs/recipes/flip_and_rotate - file: docs/recipes/pose_cfg_file_breakdown + - file: docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook - caption: Mission & Contribute chapters: - file: docs/MISSION_AND_VALUES From d9d461f82d06d92c39b7b9fc3ecdc5acb20a0133 Mon Sep 17 00:00:00 2001 From: rizarae-p Date: Wed, 14 Jun 2023 13:21:56 +0200 Subject: [PATCH 6/7] fixed hyperlinks --- ...ing_notebooks_into_the_DLC_main_cookbook.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md index 33043a1de..4466ef9ab 100644 --- a/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md +++ b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md @@ -30,7 +30,7 @@ To accomplish this, you need the following installed: ``` ## Steps -1. Double check for spelling and grammatical errors (on [Grammarly](#https://grammarly.com/) or by using Jupyter notebook's spellcheck extension called `spellchecker`). +1. Double check for spelling and grammatical errors (on Grammarly - https://grammarly.com/ or by using Jupyter notebook's spellcheck extension called `spellchecker`). *** ``` jupyter nbextension enable spellchecker/main @@ -44,7 +44,7 @@ To accomplish this, you need the following installed: jupyter nbconvert --to markdown [notebook.ipynb] ``` *** - + 3. Move your newly converted Markdown file (.md) to directory: `docs/recipes/` *** @@ -58,26 +58,26 @@ To accomplish this, you need the following installed: - file: docs/recipes/pose_cfg_file_breakdown ``` *** - + 5. Build your notebook into the DLC recipe book *** ``` - jupyter book build /absolute/path/to/the/DLC/repoDeepLabCut2023version + jupyter book build /absolute/path/to/the/DLC/repoDeepLabCut2023version ``` *** Example: *** ``` - jupyter book build /Users/rae/DeepLabCut2023version + jupyter book build /Users/rae/DeepLabCut2023version ``` *** The build log should look like below: -
    + -5. Test locally by checking the `index.html` file in `/Users/rae/Desktop/DeepLabCut2023version/_build/html/` +6. Test locally by checking the `index.html` file in `/Users/rae/Desktop/DeepLabCut2023version/_build/html/` -6. When everything is a-okay, commit to Git. If not, edit your file and go to back to step 1. +7. When everything is a-okay, commit to Git. If not, edit your file and go to back to step 1. **`git status`** to check the local changes in your current project ``` @@ -100,6 +100,6 @@ To accomplish this, you need the following installed: ``` git push ``` -5. When everything's clear, confirm your pull request on the Git website: https://github.com/DeepLabCutAIResidency/DeepLabCut2023version +8. When everything's clear, confirm your pull request on the Git website: https://github.com/DeepLabCutAIResidency/DeepLabCut2023version ## All done! From 31bb7b8b3f46b2d8e1c0b5a839d1db79b6d8d33d Mon Sep 17 00:00:00 2001 From: Mackenzie Mathis Date: Thu, 27 Jul 2023 15:19:07 +0200 Subject: [PATCH 7/7] Update publishing_notebooks_into_the_DLC_main_cookbook.md - note you can just use the docs tag! --- .../recipes/publishing_notebooks_into_the_DLC_main_cookbook.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md index 4466ef9ab..4d6479a3a 100644 --- a/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md +++ b/docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook.md @@ -15,8 +15,7 @@ To accomplish this, you need the following installed: You can do this by running the following command: *** ``` - pip install jupyter-book numpydoc nbconvert jupyter_contrib_nbextensions - jupyter contrib nbextension install --user + pip install deeplabcut[docs] ``` *** **Relevant Git repos:**