Skip to content
Snippets Groups Projects
Commit f9d3f1b2 authored by Lucia D. Hradecka's avatar Lucia D. Hradecka
Browse files

recompute speed tables with std

parent 571f399c
No related branches found
No related tags found
1 merge request!12Make version 1.3.2 default
Hide whitespace changes
Inline Side-by-side
README.md
+ 35
− 35
View file @ f9d3f1b2
...@@ -349,39 +349,39 @@ RandomCrop ...@@ -349,39 +349,39 @@ RandomCrop
### Runtime ### Runtime
Here, we present the execution times of individual transformations from our library Here, we present the execution times (mean ± standard deviation) of individual transformations from our library
with respect to input image size. with respect to input image size.
The shape (size) of inputs was [1, 32, 32, 32, 1] (32k voxels), [4, 32, 32, 32, 5] (655k voxels), The shape (size) of inputs was [1, 32, 32, 32, 1] (32k voxels), [4, 32, 32, 32, 5] (655k voxels),
[4, 64, 64, 64, 5] (5M voxels), and [4, 128, 128, 128, 5] (42M voxels), respectively. [4, 64, 64, 64, 5] (5M voxels), and [4, 128, 128, 128, 5] (42M voxels), respectively.
The runtimes, presented in milliseconds, were averaged over 100 runs. The runtimes, presented in milliseconds, were averaged over 100 runs.
All measurements were done on a single workstation with an i7-7700 CPU @ 3.60GHz. All measurements were done on a single workstation with a Ryzen 7-3700X CPU @ 3.60GHz.
| Transformation | 32k voxels | 655k voxels | 5M voxels | 42M voxels | | Transformation | 32k voxels | 655k voxels | 5M voxels | 42M voxels |
|:-------------------------|-----------:|-------------:|-------------:|------------:| |:-------------------------|-----------:|------------:|------------:|--------------:|
| AffineTransform | 3 ms | 26 ms | 113 ms | 845 ms | | AffineTransform | 2 ± 0 ms | 23 ± 2 ms | 111 ± 11 ms | 988 ± 33 ms |
| RandomAffineTransform | 2 ms | 19 ms | 110 ms | 899 ms | | RandomAffineTransform | 3 ± 1 ms | 24 ± 1 ms | 110 ± 7 ms | 1005 ± 28 ms |
| Scale | 2 ms | 19 ms | 103 ms | 854 ms | | Scale | 3 ± 1 ms | 25 ± 2 ms | 109 ± 9 ms | 990 ± 21 ms |
| RandomScale | 2 ms | 22 ms | 132 ms | 937 ms | | RandomScale | 3 ± 1 ms | 25 ± 2 ms | 111 ± 9 ms | 1000 ± 30 ms |
| Flip | < 1 ms | 1 ms | 11 ms | 86 ms | | Flip | < 1 ± 0 ms | 2 ± 0 ms | 14 ± 1 ms | 107 ± 8 ms |
| RandomFlip | < 1 ms | 1 ms | 8 ms | 66 ms | | RandomFlip | < 1 ± 0 ms | 1 ± 1 ms | 13 ± 5 ms | 90 ± 31 ms |
| RandomRotate90 | < 1 ms | 1 ms | 14 ms | 197 ms | | RandomRotate90 | < 1 ± 0 ms | 2 ± 1 ms | 18 ± 5 ms | 150 ± 44 ms |
| GaussianBlur | 1 ms | 9 ms | 82 ms | 855 ms | | GaussianBlur | 1 ± 0 ms | 13 ± 1 ms | 150 ± 3 ms | 5501 ± 148 ms |
| RandomGaussianBlur | < 1 ms | 8 ms | 74 ms | 788 ms | | RandomGaussianBlur | 1 ± 0 ms | 11 ± 1 ms | 151 ± 18 ms | 5623 ± 176 ms |
| GaussianNoise | 1 ms | 15 ms | 124 ms | 989 ms | | GaussianNoise | < 1 ± 0 ms | 10 ± 1 ms | 83 ± 3 ms | 687 ± 8 ms |
| PoissonNoise | 1 ms | 21 ms | 176 ms | 1427 ms | | PoissonNoise | 1 ± 0 ms | 20 ± 1 ms | 160 ± 1 ms | 1285 ± 18 ms |
| HistogramEqualization | 2 ms | 35 ms | 285 ms | 2330 ms | | HistogramEqualization | 2 ± 0 ms | 35 ± 1 ms | 266 ± 5 ms | 2120 ± 18 ms |
| Normalize | < 1 ms | 2 ms | 17 ms | 158 ms | | Normalize | < 1 ± 0 ms | 6 ± 1 ms | 59 ± 3 ms | 469 ± 13 ms |
| NormalizeMeanStd | < 1 ms | 1 ms | 7 ms | 58 ms | | NormalizeMeanStd | < 1 ± 0 ms | 1 ± 0 ms | 9 ± 1 ms | 69 ± 5 ms |
| RandomBrightnessContrast | < 1 ms | < 1 ms | 4 ms | 38 ms | | RandomBrightnessContrast | < 1 ± 0 ms | 1 ± 0 ms | 6 ± 1 ms | 49 ± 8 ms |
| RandomGamma | < 1 ms | 7 ms | 55 ms | 453 ms | | RandomGamma | < 1 ± 0 ms | 7 ± 0 ms | 59 ± 1 ms | 469 ± 6 ms |
### Runtime: Comparison to Other Libraries ### Runtime: Comparison to Other Libraries
We also present the execution times of eight commonly used transformations, comparing the performance We also present the execution times (mean ± standard deviation) of eight commonly used transformations,
of our `Bio-Volumentations` to other libraries capable of processing volumetric image data: comparing the performance of our `Bio-Volumentations` to other libraries capable of processing volumetric
`TorchIO` [3], `Volumentations` [4, 5], and `Gunpowder` [6]. image data: `TorchIO` [3], `Volumentations` [4, 5], and `Gunpowder` [6].
Asterisks (*) denote transformations that only partially correspond to the desired functionality. Asterisks (*) denote transformations that only partially correspond to the desired functionality.
Dashes (-) denote transformations that are missing from the respective library. Dashes (-) denote transformations that are missing from the respective library.
...@@ -390,17 +390,17 @@ The runtimes, presented in milliseconds, were averaged over 100 runs. ...@@ -390,17 +390,17 @@ The runtimes, presented in milliseconds, were averaged over 100 runs.
All measurements were done with a single-channel volumetric input image of size (256, 256, 256) All measurements were done with a single-channel volumetric input image of size (256, 256, 256)
on a single workstation with a Ryzen 7-3700X CPU @ 3.60GHz. on a single workstation with a Ryzen 7-3700X CPU @ 3.60GHz.
| Transformation | `TorchIO` | `Volumentations` | `Gunpowder` | `Bio-Volumentations` | | Transformation | `TorchIO` | `Volumentations` | `Gunpowder` | `Bio-Volumentations` |
|:-------------------------------------|---------------:|---------------------:|-------------:|---------------------:| |:-------------------------------|----------------:|-----------------:|--------------:|---------------------:|
| Cropping | *26 ms | 20 ms | **7 ms** | 20 ms | | Cropping | *27 ± 3 ms | 20 ± 3 ms | **6 ± 1 ms** | 21 ± 3 ms |
| Flipping | 48 ms | 39 ms | **31 ms** | 34 ms | | Padding | 64 ± 10 ms | **32 ± 6 ms** | 64 ± 4 ms | 43 ± 2 ms |
| Affine transform | **931 ms** | *4177 ms | - | 2719 ms | | Flipping | 50 ± 13 ms | 40 ± 10 ms | 36 ± 3 ms | **31 ± 11 ms** |
| Affine transform (anisotropic image) | - | - | - | **2723 ms** | | Affine transform | **860 ± 24 ms** | *3964 ± 816 ms | - | 2595 ± 39 ms |
| Gaussian blur | 4699 ms | - | - | **3149 ms** | | Affine transform (anisotropic) | - | - | - | **2608 ± 36 ms** |
| Gaussian noise | **182 ms** | 405 ms | *340 ms | 400 ms | | Gaussian blur | 4417 ± 103 ms | - | - | **3017 ± 84 ms** |
| Brightness and contrast change | - | 75 ms | 183 ms | **28 ms** | | Gaussian noise | **183 ± 7 ms** | 393 ± 5 ms | *425 ± 14 ms | 266 ± 8 ms |
| Padding | 68 ms | **30 ms** | 54 ms | 43 ms | | Brightness and contrast change | - | 73 ± 2 ms | 197 ± 6 ms | **29 ± 4 ms** |
| Z-normalization | 214 ms | **119 ms** | - | 133 ms | | Z-normalization | 219 ± 11 ms | **124 ± 19 ms** | - | 203 ± 25 ms |
[3] Pérez-García F, Sparks R, Ourselin S. TorchIO: A Python library for efficient loading, [3] Pérez-García F, Sparks R, Ourselin S. TorchIO: A Python library for efficient loading,
preprocessing, augmentation and patch-based sampling of medical images in deep learning. preprocessing, augmentation and patch-based sampling of medical images in deep learning.
......
tests/runtime-100_iterations-all-1.txt
+ 23
− 23
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 0.019, Average: 0.020, Maximum: 0.032. (Library: volum, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.019, Average: 20.332, Maximum: 34.008, Std: 2.643. (Library: volum, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.029, Average: 0.030, Maximum: 0.036. (Library: volum, Transform: PadIfNeeded, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.029, Average: 32.167, Maximum: 54.388, Std: 6.429. (Library: volum, Transform: PadIfNeeded, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.049, Average: 0.039, Maximum: 0.061. (Library: volum, Transform: Flip, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.037, Average: 39.565, Maximum: 77.488, Std: 9.749. (Library: volum, Transform: Flip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.675, Average: 4.177, Maximum: 5.590. (Library: volum, Transform: Composed-Rotate-RandomScale, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 4.398, Average: 3963.954, Maximum: 5609.152, Std: 815.721. (Library: volum, Transform: Composed-Rotate-RandomScale, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.398, Average: 0.405, Maximum: 0.539. (Library: volum, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.398, Average: 393.493, Maximum: 418.392, Std: 5.469. (Library: volum, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.073, Average: 0.075, Maximum: 0.109. (Library: volum, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.073, Average: 72.761, Maximum: 82.582, Std: 2.218. (Library: volum, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.112, Average: 0.119, Maximum: 0.161. (Library: volum, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.113, Average: 124.496, Maximum: 166.539, Std: 18.688. (Library: volum, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.019, Average: 0.020, Maximum: 0.033. (Library: biovol, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.020, Average: 20.567, Maximum: 34.219, Std: 3.283. (Library: biovol, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.040, Average: 0.043, Maximum: 0.071. (Library: biovol, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.041, Average: 42.664, Maximum: 50.102, Std: 2.314. (Library: biovol, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.018, Average: 0.034, Maximum: 0.078. (Library: biovol, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.018, Average: 30.892, Maximum: 50.449, Std: 10.737. (Library: biovol, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.774, Average: 2.719, Maximum: 2.822. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 2.674, Average: 2594.634, Maximum: 2657.256, Std: 38.547. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.721, Average: 2.723, Maximum: 2.770. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 2.626, Average: 2607.886, Maximum: 2737.374, Std: 36.455. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 3.139, Average: 3.149, Maximum: 3.276. (Library: biovol, Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 2.969, Average: 3017.202, Maximum: 3215.959, Std: 83.810. (Library: biovol, Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.404, Average: 0.400, Maximum: 0.410. (Library: biovol, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.262, Average: 266.162, Maximum: 291.140, Std: 7.973. (Library: biovol, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.027, Average: 0.028, Maximum: 0.046. (Library: biovol, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.027, Average: 28.546, Maximum: 47.011, Std: 4.146. (Library: biovol, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.170, Average: 0.133, Maximum: 0.173. (Library: biovol, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.193, Average: 202.717, Maximum: 279.119, Std: 25.378. (Library: biovol, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.025, Average: 0.026, Maximum: 0.041. (Library: torchio, Transform: CropOrPad, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.028, Average: 26.766, Maximum: 40.356, Std: 3.409. (Library: torchio, Transform: CropOrPad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.068, Average: 0.068, Maximum: 0.114. (Library: torchio, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.062, Average: 64.392, Maximum: 109.365, Std: 9.743. (Library: torchio, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.050, Average: 0.048, Maximum: 0.096. (Library: torchio, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.093, Average: 49.885, Maximum: 96.358, Std: 12.729. (Library: torchio, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.940, Average: 0.931, Maximum: 0.966. (Library: torchio, Transform: RandomAffine, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.877, Average: 859.602, Maximum: 915.250, Std: 24.132. (Library: torchio, Transform: RandomAffine, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.709, Average: 4.699, Maximum: 5.181. (Library: torchio, Transform: RandomBlur, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 4.379, Average: 4417.002, Maximum: 4831.463, Std: 103.232. (Library: torchio, Transform: RandomBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.295, Average: 0.182, Maximum: 0.209. (Library: torchio, Transform: RandomNoise, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.162, Average: 182.722, Maximum: 204.837, Std: 6.716. (Library: torchio, Transform: RandomNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.181, Average: 0.214, Maximum: 0.253. (Library: torchio, Transform: ZNormalization, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.183, Average: 218.836, Maximum: 258.122, Std: 11.045. (Library: torchio, Transform: ZNormalization, Iterations: 100, ImageSize: (1, 256, 256, 256))
tests/runtime-100_iterations-biovol.txt 0 → 100644
+ 64
− 0
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 2.999, Average: 2.391, Maximum: 3.011, Std: 0.488. (Transform: AffineTransform, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 23.005, Average: 22.835, Maximum: 30.007, Std: 1.613. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 122.418, Average: 110.590, Maximum: 166.709, Std: 11.004. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 979.140, Average: 988.400, Maximum: 1127.026, Std: 33.416. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 9.002, Average: 2.631, Maximum: 4.001, Std: 0.611. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 23.005, Average: 24.025, Maximum: 29.007, Std: 1.166. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 103.023, Average: 109.603, Maximum: 142.608, Std: 7.170. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 978.616, Average: 1005.322, Maximum: 1124.320, Std: 27.884. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 6.002, Average: 2.583, Maximum: 4.044, Std: 0.624. (Transform: Scale, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 34.008, Average: 24.918, Maximum: 35.008, Std: 1.831. (Transform: Scale, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 106.024, Average: 108.701, Maximum: 133.043, Std: 8.544. (Transform: Scale, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 979.041, Average: 990.209, Maximum: 1071.305, Std: 21.161. (Transform: Scale, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 2.721, Maximum: 4.040, Std: 0.609. (Transform: RandomScale, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 24.005, Average: 24.554, Maximum: 33.197, Std: 1.574. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 105.023, Average: 110.779, Maximum: 136.389, Std: 8.759. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 1009.226, Average: 1000.158, Maximum: 1155.325, Std: 30.079. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 3.000, Average: 0.020, Maximum: 1.001, Std: 0.140. (Transform: Flip, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 2.001, Average: 1.550, Maximum: 2.001, Std: 0.498. (Transform: Flip, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 14.003, Average: 13.733, Maximum: 16.075, Std: 0.678. (Transform: Flip, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 99.486, Average: 107.409, Maximum: 143.800, Std: 7.644. (Transform: Flip, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.080, Maximum: 1.000, Std: 0.271. (Transform: RandomFlip, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 1.001, Average: 1.250, Maximum: 2.002, Std: 0.638. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 8.001, Average: 12.588, Maximum: 22.227, Std: 4.520. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 116.957, Average: 90.360, Maximum: 146.485, Std: 30.669. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.040, Maximum: 1.000, Std: 0.196. (Transform: RandomRotate90, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 2.000, Average: 1.684, Maximum: 4.002, Std: 0.631. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 19.004, Average: 17.717, Maximum: 42.351, Std: 4.915. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 95.563, Average: 150.484, Maximum: 241.054, Std: 44.266. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 6.001, Average: 0.650, Maximum: 1.009, Std: 0.477. (Transform: GaussianBlur, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 13.003, Average: 12.913, Maximum: 15.004, Std: 0.680. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 144.032, Average: 149.573, Maximum: 159.172, Std: 2.953. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 5384.104, Average: 5501.425, Maximum: 6005.390, Std: 147.510. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 8.002, Average: 0.640, Maximum: 1.007, Std: 0.480. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 11.002, Average: 11.455, Maximum: 14.003, Std: 1.150. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 135.303, Average: 150.725, Maximum: 201.741, Std: 18.470. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 5544.291, Average: 5623.328, Maximum: 6251.696, Std: 175.690. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 8.002, Average: 0.410, Maximum: 1.001, Std: 0.492. (Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 10.002, Average: 10.332, Maximum: 12.003, Std: 0.649. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 82.018, Average: 83.341, Maximum: 93.687, Std: 3.010. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 644.610, Average: 686.519, Maximum: 709.856, Std: 8.110. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.000, Average: 0.834, Maximum: 1.033, Std: 0.378. (Transform: PoissonNoise, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 19.004, Average: 19.558, Maximum: 23.216, Std: 0.943. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 158.035, Average: 160.499, Maximum: 164.811, Std: 1.388. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 1298.296, Average: 1284.684, Maximum: 1353.489, Std: 17.769. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 1.642, Maximum: 2.049, Std: 0.486. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 36.008, Average: 34.615, Maximum: 40.379, Std: 1.441. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 262.933, Average: 265.706, Maximum: 296.106, Std: 4.961. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 2084.197, Average: 2120.427, Maximum: 2227.009, Std: 18.423. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.002, Average: 0.210, Maximum: 1.001, Std: 0.407. (Transform: Normalize, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 5.000, Average: 5.546, Maximum: 6.060, Std: 0.501. (Transform: Normalize, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 56.012, Average: 59.426, Maximum: 73.612, Std: 3.251. (Transform: Normalize, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 472.319, Average: 468.856, Maximum: 528.517, Std: 12.674. (Transform: Normalize, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 0.010, Maximum: 1.000, Std: 0.100. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 0.000, Average: 0.750, Maximum: 1.001, Std: 0.433. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 8.001, Average: 8.606, Maximum: 11.182, Std: 0.802. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 64.169, Average: 69.187, Maximum: 84.799, Std: 4.649. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.040, Maximum: 1.000, Std: 0.196. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 1.001, Average: 0.600, Maximum: 1.002, Std: 0.490. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 6.002, Average: 6.266, Maximum: 9.086, Std: 0.818. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 41.009, Average: 48.784, Maximum: 87.520, Std: 7.793. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.002, Average: 0.350, Maximum: 1.001, Std: 0.477. (Transform: RandomGamma, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 7.000, Average: 6.912, Maximum: 8.002, Std: 0.402. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 59.252, Average: 59.024, Maximum: 61.537, Std: 0.769. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 459.582, Average: 469.455, Maximum: 487.058, Std: 5.654. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
tests/runtime-100_iterations-gunpowder.txt
+ 5
− 5
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 0.008, Average: 0.007, Maximum: 0.037. (Library: gunpowder, Transform: Composed-Crop-RandomLocation, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.007, Average: 6.220, Maximum: 7.100, Std: 0.513. (Library: gunpowder, Transform: Composed-Crop-RandomLocation, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.051, Average: 0.054, Maximum: 0.084. (Library: gunpowder, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.069, Average: 63.710, Maximum: 66.549, Std: 3.555. (Library: gunpowder, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.031, Average: 0.031, Maximum: 0.051. (Library: gunpowder, Transform: SimpleAugment, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.030, Average: 35.789, Maximum: 38.291, Std: 2.614. (Library: gunpowder, Transform: SimpleAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.338, Average: 0.340, Maximum: 0.371. (Library: gunpowder, Transform: NoiseAugment, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.424, Average: 425.064, Maximum: 486.558, Std: 14.440. (Library: gunpowder, Transform: NoiseAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.171, Average: 0.183, Maximum: 0.235. (Library: gunpowder, Transform: IntensityAugment, Iterations: 100, ImageSize: (1, 256, 256, 256)) Runtime in seconds. FirstRun: 0.205, Average: 197.365, Maximum: 208.468, Std: 5.624. (Library: gunpowder, Transform: IntensityAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
tests/speed_test_biovol.py
+ 33
− 43
View file @ f9d3f1b2
...@@ -32,42 +32,35 @@ from src.core.composition import * ...@@ -32,42 +32,35 @@ from src.core.composition import *
from src.augmentations import * from src.augmentations import *
# size_sample = [(1, 256, 256, 256), (3, 256, 256, 256), (1, 256, 256, 256, 10), (3, 256, 256, 256, 10)] size_sample = [(1, 32, 32, 32, 1), (4, 32, 32, 32, 5), (4, 64, 64, 64, 5), (4, 128, 128, 128, 5)]
size_sample = [(1, 128, 128, 128)] # size_sample = [(1, 32, 32, 32)]
# num_repeat = 100 num_repeat = 100
num_repeat = 10 # num_repeat = 10
augmentations_to_check = [ augmentations_to_check = [
# RandomAffineTransform(angle_limit=[22.5, 22.5, 22.5], p=1), AffineTransform(angles=(22.5, 22.5, 22.5), translation=(20, 20, 20), scale=(0.8, 0.8, 0.8), p=1),
RandomAffineTransform(angle_limit=[22.5, 22.5, 22.5], RandomAffineTransform(angle_limit=(22.5, 22.5, 22.5), translation_limit=(20, 20, 20),
translation_limit=[10, 10, 10], scaling_limit=(0.8, 0.8, 0.8), p=1),
scaling_limit=[.2, .2, .2], Scale(scales=(0.8, 1, 1.25), p=1),
spacing=[1, 0.5, 2], RandomScale(scaling_limit=(0.8, 1.25), p=1),
p=1), Flip(p=1),
# RandomScale(scaling_limit=(0.75, 1), p=1), RandomFlip(p=1),
# RandomScale(scaling_limit=(1, 1.5), p=1), RandomRotate90(p=1),
Scale(scales=0.75, p=1), GaussianBlur(p=1),
Scale(scales=1.5, p=1), RandomGaussianBlur(p=1),
Flip(axes=[1, 2, 3], p=1), GaussianNoise(p=1),
# GaussianBlur(sigma=0, p=1), PoissonNoise(p=1),
# GaussianNoise(var_limit=(0.001, 0.1), mean=0, p=1), HistogramEqualization(p=1),
# HistogramEqualization(bins=256, p=1), Normalize(p=1),
# Normalize(mean=0, std=1, p=1), NormalizeMeanStd(mean=0.1, std=1, p=1),
# NormalizeMeanStd(mean=0.1, std=1, p=1), RandomBrightnessContrast(p=0.1),
# RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.1), RandomGamma(p=1),
# RandomFlip(axes_to_choose=None, p=1),
# RandomGamma(gamma_limit=(0.8, 1.2), p=1),
# RandomGaussianBlur(max_sigma=0.8, p=1),
# RandomRotate90(axes=[1, 2, 3] , p=1),
# Scale(scale_factor=1.5, p=1),
# Scale(scale_factor=0.75, p=1),
] ]
def single_transform(iterations, size, augmentation): def single_transform(iterations, size, augmentation):
cumulative = 0 times = np.zeros(iterations)
maximum = 0
for i in range(iterations): for i in range(iterations):
# get data and transformation # get data and transformation
test = np.random.uniform(low=0, high=1, size=size) test = np.random.uniform(low=0, high=1, size=size)
...@@ -77,23 +70,18 @@ def single_transform(iterations, size, augmentation): ...@@ -77,23 +70,18 @@ def single_transform(iterations, size, augmentation):
# run and measure # run and measure
second_time = time.time() second_time = time.time()
aug_data = aug(**data) aug_data = aug(**data)
_ = aug_data['image'].shape
time_spent = time.time() - second_time time_spent = time.time() - second_time
_ = aug_data['image'].shape
times[i] = time_spent * 1000 # convert from seconds to milliseconds
# accumulate the results return times
cumulative += time_spent
if time_spent > maximum:
maximum = time_spent
return maximum, cumulative
def transformation_speed_benchmark(iterations): def transformation_speed_benchmark(iterations):
f = open(f'./runtime-{num_repeat}_iterations.txt', 'w') f = open(f'./runtime-{num_repeat}_iterations-biovol.txt', 'w')
for i, augmentation in enumerate(augmentations_to_check): # random_scale_transform for i, augmentation in enumerate(augmentations_to_check):
# augmentation = augmentation_getter(augmentations_to_check, i, size)
aug_name = augmentation.__class__.__name__ aug_name = augmentation.__class__.__name__
print(aug_name) print(aug_name)
...@@ -107,13 +95,15 @@ def transformation_speed_benchmark(iterations): ...@@ -107,13 +95,15 @@ def transformation_speed_benchmark(iterations):
# run for the first time to allocate memory etc. # run for the first time to allocate memory etc.
first_time = time.time() first_time = time.time()
aug_data = aug(**data) aug_data = aug(**data)
first_result = time.time() - first_time first_result = (time.time() - first_time) * 1000 # convert to milliseconds
# run the measured runs # run the measured runs
maximum, cumulative = single_transform(iterations, size, augmentation) times_arr = single_transform(iterations, size, augmentation)
result_time = cumulative / iterations
# log
log_message = f'Runtime in seconds. ' \ log_message = f'Runtime in seconds. ' \
f'FirstRun: {first_result:.3f}, Average: {result_time:.3f}, Maximum: {maximum:.3f}. ' \ f'FirstRun: {first_result:.3f}, Average: {times_arr.mean():.3f}, ' \
f'Maximum: {times_arr.max():.3f}, Std: {times_arr.std():.3f}. ' \
f'(Transform: {aug_name}, Iterations: {iterations}, ImageSize: {size})\n' f'(Transform: {aug_name}, Iterations: {iterations}, ImageSize: {size})\n'
f.write(log_message) f.write(log_message)
print(log_message) print(log_message)
......
tests/speed_test_other.py
+ 8
− 14
View file @ f9d3f1b2
...@@ -249,10 +249,8 @@ def transform_data(lib, data, pipeline): ...@@ -249,10 +249,8 @@ def transform_data(lib, data, pipeline):
pass pass
def single_transform(iterations, shape, augmentation, lib): def single_transform(iterations, shape, augmentation, lib) -> np.ndarray:
cumulative = 0 times = np.zeros(iterations)
maximum = 0
for i in range(iterations): for i in range(iterations):
# prepare data and transformation pipeline # prepare data and transformation pipeline
transformation_pipeline = init_compose(lib, augmentation) transformation_pipeline = init_compose(lib, augmentation)
...@@ -262,13 +260,9 @@ def single_transform(iterations, shape, augmentation, lib): ...@@ -262,13 +260,9 @@ def single_transform(iterations, shape, augmentation, lib):
t_0 = time.time() t_0 = time.time()
_ = transform_data(lib, data, transformation_pipeline) _ = transform_data(lib, data, transformation_pipeline)
time_spent = time.time() - t_0 time_spent = time.time() - t_0
times[i] = time_spent * 1000 # convert from seconds to milliseconds
# accumulate time return times
cumulative += time_spent
if time_spent > maximum:
maximum = time_spent
return maximum, cumulative
def transformation_speed_benchmark(iterations): def transformation_speed_benchmark(iterations):
...@@ -290,15 +284,15 @@ def transformation_speed_benchmark(iterations): ...@@ -290,15 +284,15 @@ def transformation_speed_benchmark(iterations):
data = get_input_data(lib, shape) data = get_input_data(lib, shape)
first_time = time.time() first_time = time.time()
_ = transform_data(lib, data, transformation_pipeline) _ = transform_data(lib, data, transformation_pipeline)
first_result = time.time() - first_time first_result = time.time() - first_time # this is in seconds !!!
# the measured runs # the measured runs
maximum, cumulative = single_transform(iterations, shape, augmentation, lib) times_arr = single_transform(iterations, shape, augmentation, lib) # this is in milliseconds
result_time = cumulative / iterations
# logging # logging
log_message = f'Runtime in seconds. ' \ log_message = f'Runtime in seconds. ' \
f'FirstRun: {first_result:.3f}, Average: {result_time:.3f}, Maximum: {maximum:.3f}. ' \ f'FirstRun: {first_result:.3f}, Average: {times_arr.mean():.3f}, ' \
f'Maximum: {times_arr.max():.3f}, Std: {times_arr.std():.3f}. ' \
f'(Library: {lib}, Transform: {aug_name}, Iterations: {iterations}, ImageSize: {shape})\n' f'(Library: {lib}, Transform: {aug_name}, Iterations: {iterations}, ImageSize: {shape})\n'
f.write(log_message) f.write(log_message)
print(log_message) print(log_message)
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment