Skip to content
Snippets Groups Projects
Commit f9d3f1b2 authored by Lucia D. Hradecka's avatar Lucia D. Hradecka
Browse files

recompute speed tables with std

parent 571f399c
No related branches found
No related tags found
1 merge request!12Make version 1.3.2 default
Hide whitespace changes
Inline Side-by-side
README.md
+ 35
− 35
View file @ f9d3f1b2
......@@ -349,39 +349,39 @@ RandomCrop
### Runtime
Here, we present the execution times of individual transformations from our library
Here, we present the execution times (mean ± standard deviation) of individual transformations from our library
with respect to input image size.
The shape (size) of inputs was [1, 32, 32, 32, 1] (32k voxels), [4, 32, 32, 32, 5] (655k voxels),
[4, 64, 64, 64, 5] (5M voxels), and [4, 128, 128, 128, 5] (42M voxels), respectively.
The runtimes, presented in milliseconds, were averaged over 100 runs.
All measurements were done on a single workstation with an i7-7700 CPU @ 3.60GHz.
| Transformation | 32k voxels | 655k voxels | 5M voxels | 42M voxels |
|:-------------------------|-----------:|-------------:|-------------:|------------:|
| AffineTransform | 3 ms | 26 ms | 113 ms | 845 ms |
| RandomAffineTransform | 2 ms | 19 ms | 110 ms | 899 ms |
| Scale | 2 ms | 19 ms | 103 ms | 854 ms |
| RandomScale | 2 ms | 22 ms | 132 ms | 937 ms |
| Flip | < 1 ms | 1 ms | 11 ms | 86 ms |
| RandomFlip | < 1 ms | 1 ms | 8 ms | 66 ms |
| RandomRotate90 | < 1 ms | 1 ms | 14 ms | 197 ms |
| GaussianBlur | 1 ms | 9 ms | 82 ms | 855 ms |
| RandomGaussianBlur | < 1 ms | 8 ms | 74 ms | 788 ms |
| GaussianNoise | 1 ms | 15 ms | 124 ms | 989 ms |
| PoissonNoise | 1 ms | 21 ms | 176 ms | 1427 ms |
| HistogramEqualization | 2 ms | 35 ms | 285 ms | 2330 ms |
| Normalize | < 1 ms | 2 ms | 17 ms | 158 ms |
| NormalizeMeanStd | < 1 ms | 1 ms | 7 ms | 58 ms |
| RandomBrightnessContrast | < 1 ms | < 1 ms | 4 ms | 38 ms |
| RandomGamma | < 1 ms | 7 ms | 55 ms | 453 ms |
All measurements were done on a single workstation with a Ryzen 7-3700X CPU @ 3.60GHz.
| Transformation | 32k voxels | 655k voxels | 5M voxels | 42M voxels |
|:-------------------------|-----------:|------------:|------------:|--------------:|
| AffineTransform | 2 ± 0 ms | 23 ± 2 ms | 111 ± 11 ms | 988 ± 33 ms |
| RandomAffineTransform | 3 ± 1 ms | 24 ± 1 ms | 110 ± 7 ms | 1005 ± 28 ms |
| Scale | 3 ± 1 ms | 25 ± 2 ms | 109 ± 9 ms | 990 ± 21 ms |
| RandomScale | 3 ± 1 ms | 25 ± 2 ms | 111 ± 9 ms | 1000 ± 30 ms |
| Flip | < 1 ± 0 ms | 2 ± 0 ms | 14 ± 1 ms | 107 ± 8 ms |
| RandomFlip | < 1 ± 0 ms | 1 ± 1 ms | 13 ± 5 ms | 90 ± 31 ms |
| RandomRotate90 | < 1 ± 0 ms | 2 ± 1 ms | 18 ± 5 ms | 150 ± 44 ms |
| GaussianBlur | 1 ± 0 ms | 13 ± 1 ms | 150 ± 3 ms | 5501 ± 148 ms |
| RandomGaussianBlur | 1 ± 0 ms | 11 ± 1 ms | 151 ± 18 ms | 5623 ± 176 ms |
| GaussianNoise | < 1 ± 0 ms | 10 ± 1 ms | 83 ± 3 ms | 687 ± 8 ms |
| PoissonNoise | 1 ± 0 ms | 20 ± 1 ms | 160 ± 1 ms | 1285 ± 18 ms |
| HistogramEqualization | 2 ± 0 ms | 35 ± 1 ms | 266 ± 5 ms | 2120 ± 18 ms |
| Normalize | < 1 ± 0 ms | 6 ± 1 ms | 59 ± 3 ms | 469 ± 13 ms |
| NormalizeMeanStd | < 1 ± 0 ms | 1 ± 0 ms | 9 ± 1 ms | 69 ± 5 ms |
| RandomBrightnessContrast | < 1 ± 0 ms | 1 ± 0 ms | 6 ± 1 ms | 49 ± 8 ms |
| RandomGamma | < 1 ± 0 ms | 7 ± 0 ms | 59 ± 1 ms | 469 ± 6 ms |
### Runtime: Comparison to Other Libraries
We also present the execution times of eight commonly used transformations, comparing the performance
of our `Bio-Volumentations` to other libraries capable of processing volumetric image data:
`TorchIO` [3], `Volumentations` [4, 5], and `Gunpowder` [6].
We also present the execution times (mean ± standard deviation) of eight commonly used transformations,
comparing the performance of our `Bio-Volumentations` to other libraries capable of processing volumetric
image data: `TorchIO` [3], `Volumentations` [4, 5], and `Gunpowder` [6].
Asterisks (*) denote transformations that only partially correspond to the desired functionality.
Dashes (-) denote transformations that are missing from the respective library.
......@@ -390,17 +390,17 @@ The runtimes, presented in milliseconds, were averaged over 100 runs.
All measurements were done with a single-channel volumetric input image of size (256, 256, 256)
on a single workstation with a Ryzen 7-3700X CPU @ 3.60GHz.
| Transformation | `TorchIO` | `Volumentations` | `Gunpowder` | `Bio-Volumentations` |
|:-------------------------------------|---------------:|---------------------:|-------------:|---------------------:|
| Cropping | *26 ms | 20 ms | **7 ms** | 20 ms |
| Flipping | 48 ms | 39 ms | **31 ms** | 34 ms |
| Affine transform | **931 ms** | *4177 ms | - | 2719 ms |
| Affine transform (anisotropic image) | - | - | - | **2723 ms** |
| Gaussian blur | 4699 ms | - | - | **3149 ms** |
| Gaussian noise | **182 ms** | 405 ms | *340 ms | 400 ms |
| Brightness and contrast change | - | 75 ms | 183 ms | **28 ms** |
| Padding | 68 ms | **30 ms** | 54 ms | 43 ms |
| Z-normalization | 214 ms | **119 ms** | - | 133 ms |
| Transformation | `TorchIO` | `Volumentations` | `Gunpowder` | `Bio-Volumentations` |
|:-------------------------------|----------------:|-----------------:|--------------:|---------------------:|
| Cropping | *27 ± 3 ms | 20 ± 3 ms | **6 ± 1 ms** | 21 ± 3 ms |
| Padding | 64 ± 10 ms | **32 ± 6 ms** | 64 ± 4 ms | 43 ± 2 ms |
| Flipping | 50 ± 13 ms | 40 ± 10 ms | 36 ± 3 ms | **31 ± 11 ms** |
| Affine transform | **860 ± 24 ms** | *3964 ± 816 ms | - | 2595 ± 39 ms |
| Affine transform (anisotropic) | - | - | - | **2608 ± 36 ms** |
| Gaussian blur | 4417 ± 103 ms | - | - | **3017 ± 84 ms** |
| Gaussian noise | **183 ± 7 ms** | 393 ± 5 ms | *425 ± 14 ms | 266 ± 8 ms |
| Brightness and contrast change | - | 73 ± 2 ms | 197 ± 6 ms | **29 ± 4 ms** |
| Z-normalization | 219 ± 11 ms | **124 ± 19 ms** | - | 203 ± 25 ms |
[3] Pérez-García F, Sparks R, Ourselin S. TorchIO: A Python library for efficient loading,
preprocessing, augmentation and patch-based sampling of medical images in deep learning.
......
tests/runtime-100_iterations-all-1.txt
+ 23
− 23
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 0.019, Average: 0.020, Maximum: 0.032. (Library: volum, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.029, Average: 0.030, Maximum: 0.036. (Library: volum, Transform: PadIfNeeded, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.049, Average: 0.039, Maximum: 0.061. (Library: volum, Transform: Flip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.675, Average: 4.177, Maximum: 5.590. (Library: volum, Transform: Composed-Rotate-RandomScale, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.398, Average: 0.405, Maximum: 0.539. (Library: volum, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.073, Average: 0.075, Maximum: 0.109. (Library: volum, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.112, Average: 0.119, Maximum: 0.161. (Library: volum, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.019, Average: 0.020, Maximum: 0.033. (Library: biovol, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.040, Average: 0.043, Maximum: 0.071. (Library: biovol, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.018, Average: 0.034, Maximum: 0.078. (Library: biovol, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.774, Average: 2.719, Maximum: 2.822. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.721, Average: 2.723, Maximum: 2.770. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 3.139, Average: 3.149, Maximum: 3.276. (Library: biovol, Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.404, Average: 0.400, Maximum: 0.410. (Library: biovol, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.027, Average: 0.028, Maximum: 0.046. (Library: biovol, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.170, Average: 0.133, Maximum: 0.173. (Library: biovol, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.025, Average: 0.026, Maximum: 0.041. (Library: torchio, Transform: CropOrPad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.068, Average: 0.068, Maximum: 0.114. (Library: torchio, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.050, Average: 0.048, Maximum: 0.096. (Library: torchio, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.940, Average: 0.931, Maximum: 0.966. (Library: torchio, Transform: RandomAffine, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.709, Average: 4.699, Maximum: 5.181. (Library: torchio, Transform: RandomBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.295, Average: 0.182, Maximum: 0.209. (Library: torchio, Transform: RandomNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.181, Average: 0.214, Maximum: 0.253. (Library: torchio, Transform: ZNormalization, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.019, Average: 20.332, Maximum: 34.008, Std: 2.643. (Library: volum, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.029, Average: 32.167, Maximum: 54.388, Std: 6.429. (Library: volum, Transform: PadIfNeeded, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.037, Average: 39.565, Maximum: 77.488, Std: 9.749. (Library: volum, Transform: Flip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.398, Average: 3963.954, Maximum: 5609.152, Std: 815.721. (Library: volum, Transform: Composed-Rotate-RandomScale, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.398, Average: 393.493, Maximum: 418.392, Std: 5.469. (Library: volum, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.073, Average: 72.761, Maximum: 82.582, Std: 2.218. (Library: volum, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.113, Average: 124.496, Maximum: 166.539, Std: 18.688. (Library: volum, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.020, Average: 20.567, Maximum: 34.219, Std: 3.283. (Library: biovol, Transform: RandomCrop, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.041, Average: 42.664, Maximum: 50.102, Std: 2.314. (Library: biovol, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.018, Average: 30.892, Maximum: 50.449, Std: 10.737. (Library: biovol, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.674, Average: 2594.634, Maximum: 2657.256, Std: 38.547. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.626, Average: 2607.886, Maximum: 2737.374, Std: 36.455. (Library: biovol, Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 2.969, Average: 3017.202, Maximum: 3215.959, Std: 83.810. (Library: biovol, Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.262, Average: 266.162, Maximum: 291.140, Std: 7.973. (Library: biovol, Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.027, Average: 28.546, Maximum: 47.011, Std: 4.146. (Library: biovol, Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.193, Average: 202.717, Maximum: 279.119, Std: 25.378. (Library: biovol, Transform: Normalize, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.028, Average: 26.766, Maximum: 40.356, Std: 3.409. (Library: torchio, Transform: CropOrPad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.062, Average: 64.392, Maximum: 109.365, Std: 9.743. (Library: torchio, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.093, Average: 49.885, Maximum: 96.358, Std: 12.729. (Library: torchio, Transform: RandomFlip, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.877, Average: 859.602, Maximum: 915.250, Std: 24.132. (Library: torchio, Transform: RandomAffine, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 4.379, Average: 4417.002, Maximum: 4831.463, Std: 103.232. (Library: torchio, Transform: RandomBlur, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.162, Average: 182.722, Maximum: 204.837, Std: 6.716. (Library: torchio, Transform: RandomNoise, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.183, Average: 218.836, Maximum: 258.122, Std: 11.045. (Library: torchio, Transform: ZNormalization, Iterations: 100, ImageSize: (1, 256, 256, 256))
tests/runtime-100_iterations-biovol.txt 0 → 100644
+ 64
− 0
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 2.999, Average: 2.391, Maximum: 3.011, Std: 0.488. (Transform: AffineTransform, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 23.005, Average: 22.835, Maximum: 30.007, Std: 1.613. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 122.418, Average: 110.590, Maximum: 166.709, Std: 11.004. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 979.140, Average: 988.400, Maximum: 1127.026, Std: 33.416. (Transform: AffineTransform, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 9.002, Average: 2.631, Maximum: 4.001, Std: 0.611. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 23.005, Average: 24.025, Maximum: 29.007, Std: 1.166. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 103.023, Average: 109.603, Maximum: 142.608, Std: 7.170. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 978.616, Average: 1005.322, Maximum: 1124.320, Std: 27.884. (Transform: RandomAffineTransform, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 6.002, Average: 2.583, Maximum: 4.044, Std: 0.624. (Transform: Scale, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 34.008, Average: 24.918, Maximum: 35.008, Std: 1.831. (Transform: Scale, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 106.024, Average: 108.701, Maximum: 133.043, Std: 8.544. (Transform: Scale, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 979.041, Average: 990.209, Maximum: 1071.305, Std: 21.161. (Transform: Scale, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 2.721, Maximum: 4.040, Std: 0.609. (Transform: RandomScale, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 24.005, Average: 24.554, Maximum: 33.197, Std: 1.574. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 105.023, Average: 110.779, Maximum: 136.389, Std: 8.759. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 1009.226, Average: 1000.158, Maximum: 1155.325, Std: 30.079. (Transform: RandomScale, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 3.000, Average: 0.020, Maximum: 1.001, Std: 0.140. (Transform: Flip, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 2.001, Average: 1.550, Maximum: 2.001, Std: 0.498. (Transform: Flip, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 14.003, Average: 13.733, Maximum: 16.075, Std: 0.678. (Transform: Flip, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 99.486, Average: 107.409, Maximum: 143.800, Std: 7.644. (Transform: Flip, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.080, Maximum: 1.000, Std: 0.271. (Transform: RandomFlip, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 1.001, Average: 1.250, Maximum: 2.002, Std: 0.638. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 8.001, Average: 12.588, Maximum: 22.227, Std: 4.520. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 116.957, Average: 90.360, Maximum: 146.485, Std: 30.669. (Transform: RandomFlip, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.040, Maximum: 1.000, Std: 0.196. (Transform: RandomRotate90, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 2.000, Average: 1.684, Maximum: 4.002, Std: 0.631. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 19.004, Average: 17.717, Maximum: 42.351, Std: 4.915. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 95.563, Average: 150.484, Maximum: 241.054, Std: 44.266. (Transform: RandomRotate90, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 6.001, Average: 0.650, Maximum: 1.009, Std: 0.477. (Transform: GaussianBlur, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 13.003, Average: 12.913, Maximum: 15.004, Std: 0.680. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 144.032, Average: 149.573, Maximum: 159.172, Std: 2.953. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 5384.104, Average: 5501.425, Maximum: 6005.390, Std: 147.510. (Transform: GaussianBlur, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 8.002, Average: 0.640, Maximum: 1.007, Std: 0.480. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 11.002, Average: 11.455, Maximum: 14.003, Std: 1.150. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 135.303, Average: 150.725, Maximum: 201.741, Std: 18.470. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 5544.291, Average: 5623.328, Maximum: 6251.696, Std: 175.690. (Transform: RandomGaussianBlur, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 8.002, Average: 0.410, Maximum: 1.001, Std: 0.492. (Transform: GaussianNoise, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 10.002, Average: 10.332, Maximum: 12.003, Std: 0.649. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 82.018, Average: 83.341, Maximum: 93.687, Std: 3.010. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 644.610, Average: 686.519, Maximum: 709.856, Std: 8.110. (Transform: GaussianNoise, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.000, Average: 0.834, Maximum: 1.033, Std: 0.378. (Transform: PoissonNoise, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 19.004, Average: 19.558, Maximum: 23.216, Std: 0.943. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 158.035, Average: 160.499, Maximum: 164.811, Std: 1.388. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 1298.296, Average: 1284.684, Maximum: 1353.489, Std: 17.769. (Transform: PoissonNoise, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 1.642, Maximum: 2.049, Std: 0.486. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 36.008, Average: 34.615, Maximum: 40.379, Std: 1.441. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 262.933, Average: 265.706, Maximum: 296.106, Std: 4.961. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 2084.197, Average: 2120.427, Maximum: 2227.009, Std: 18.423. (Transform: HistogramEqualization, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.002, Average: 0.210, Maximum: 1.001, Std: 0.407. (Transform: Normalize, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 5.000, Average: 5.546, Maximum: 6.060, Std: 0.501. (Transform: Normalize, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 56.012, Average: 59.426, Maximum: 73.612, Std: 3.251. (Transform: Normalize, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 472.319, Average: 468.856, Maximum: 528.517, Std: 12.674. (Transform: Normalize, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 7.002, Average: 0.010, Maximum: 1.000, Std: 0.100. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 0.000, Average: 0.750, Maximum: 1.001, Std: 0.433. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 8.001, Average: 8.606, Maximum: 11.182, Std: 0.802. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 64.169, Average: 69.187, Maximum: 84.799, Std: 4.649. (Transform: NormalizeMeanStd, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.001, Average: 0.040, Maximum: 1.000, Std: 0.196. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 1.001, Average: 0.600, Maximum: 1.002, Std: 0.490. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 6.002, Average: 6.266, Maximum: 9.086, Std: 0.818. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 41.009, Average: 48.784, Maximum: 87.520, Std: 7.793. (Transform: RandomBrightnessContrast, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
Runtime in seconds. FirstRun: 5.002, Average: 0.350, Maximum: 1.001, Std: 0.477. (Transform: RandomGamma, Iterations: 100, ImageSize: (1, 32, 32, 32, 1))
Runtime in seconds. FirstRun: 7.000, Average: 6.912, Maximum: 8.002, Std: 0.402. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 32, 32, 32, 5))
Runtime in seconds. FirstRun: 59.252, Average: 59.024, Maximum: 61.537, Std: 0.769. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 64, 64, 64, 5))
Runtime in seconds. FirstRun: 459.582, Average: 469.455, Maximum: 487.058, Std: 5.654. (Transform: RandomGamma, Iterations: 100, ImageSize: (4, 128, 128, 128, 5))
tests/runtime-100_iterations-gunpowder.txt
+ 5
− 5
View file @ f9d3f1b2
Runtime in seconds. FirstRun: 0.008, Average: 0.007, Maximum: 0.037. (Library: gunpowder, Transform: Composed-Crop-RandomLocation, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.051, Average: 0.054, Maximum: 0.084. (Library: gunpowder, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.031, Average: 0.031, Maximum: 0.051. (Library: gunpowder, Transform: SimpleAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.338, Average: 0.340, Maximum: 0.371. (Library: gunpowder, Transform: NoiseAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.171, Average: 0.183, Maximum: 0.235. (Library: gunpowder, Transform: IntensityAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.007, Average: 6.220, Maximum: 7.100, Std: 0.513. (Library: gunpowder, Transform: Composed-Crop-RandomLocation, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.069, Average: 63.710, Maximum: 66.549, Std: 3.555. (Library: gunpowder, Transform: Pad, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.030, Average: 35.789, Maximum: 38.291, Std: 2.614. (Library: gunpowder, Transform: SimpleAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.424, Average: 425.064, Maximum: 486.558, Std: 14.440. (Library: gunpowder, Transform: NoiseAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
Runtime in seconds. FirstRun: 0.205, Average: 197.365, Maximum: 208.468, Std: 5.624. (Library: gunpowder, Transform: IntensityAugment, Iterations: 100, ImageSize: (1, 256, 256, 256))
tests/speed_test_biovol.py
+ 33
− 43
View file @ f9d3f1b2
......@@ -32,42 +32,35 @@ from src.core.composition import *
from src.augmentations import *
# size_sample = [(1, 256, 256, 256), (3, 256, 256, 256), (1, 256, 256, 256, 10), (3, 256, 256, 256, 10)]
size_sample = [(1, 128, 128, 128)]
size_sample = [(1, 32, 32, 32, 1), (4, 32, 32, 32, 5), (4, 64, 64, 64, 5), (4, 128, 128, 128, 5)]
# size_sample = [(1, 32, 32, 32)]
# num_repeat = 100
num_repeat = 10
num_repeat = 100
# num_repeat = 10
augmentations_to_check = [
# RandomAffineTransform(angle_limit=[22.5, 22.5, 22.5], p=1),
RandomAffineTransform(angle_limit=[22.5, 22.5, 22.5],
translation_limit=[10, 10, 10],
scaling_limit=[.2, .2, .2],
spacing=[1, 0.5, 2],
p=1),
# RandomScale(scaling_limit=(0.75, 1), p=1),
# RandomScale(scaling_limit=(1, 1.5), p=1),
Scale(scales=0.75, p=1),
Scale(scales=1.5, p=1),
Flip(axes=[1, 2, 3], p=1),
# GaussianBlur(sigma=0, p=1),
# GaussianNoise(var_limit=(0.001, 0.1), mean=0, p=1),
# HistogramEqualization(bins=256, p=1),
# Normalize(mean=0, std=1, p=1),
# NormalizeMeanStd(mean=0.1, std=1, p=1),
# RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.1),
# RandomFlip(axes_to_choose=None, p=1),
# RandomGamma(gamma_limit=(0.8, 1.2), p=1),
# RandomGaussianBlur(max_sigma=0.8, p=1),
# RandomRotate90(axes=[1, 2, 3] , p=1),
# Scale(scale_factor=1.5, p=1),
# Scale(scale_factor=0.75, p=1),
AffineTransform(angles=(22.5, 22.5, 22.5), translation=(20, 20, 20), scale=(0.8, 0.8, 0.8), p=1),
RandomAffineTransform(angle_limit=(22.5, 22.5, 22.5), translation_limit=(20, 20, 20),
scaling_limit=(0.8, 0.8, 0.8), p=1),
Scale(scales=(0.8, 1, 1.25), p=1),
RandomScale(scaling_limit=(0.8, 1.25), p=1),
Flip(p=1),
RandomFlip(p=1),
RandomRotate90(p=1),
GaussianBlur(p=1),
RandomGaussianBlur(p=1),
GaussianNoise(p=1),
PoissonNoise(p=1),
HistogramEqualization(p=1),
Normalize(p=1),
NormalizeMeanStd(mean=0.1, std=1, p=1),
RandomBrightnessContrast(p=0.1),
RandomGamma(p=1),
]
def single_transform(iterations, size, augmentation):
cumulative = 0
maximum = 0
times = np.zeros(iterations)
for i in range(iterations):
# get data and transformation
test = np.random.uniform(low=0, high=1, size=size)
......@@ -77,23 +70,18 @@ def single_transform(iterations, size, augmentation):
# run and measure
second_time = time.time()
aug_data = aug(**data)
_ = aug_data['image'].shape
time_spent = time.time() - second_time
_ = aug_data['image'].shape
times[i] = time_spent * 1000 # convert from seconds to milliseconds
# accumulate the results
cumulative += time_spent
if time_spent > maximum:
maximum = time_spent
return maximum, cumulative
return times
def transformation_speed_benchmark(iterations):
f = open(f'./runtime-{num_repeat}_iterations.txt', 'w')
f = open(f'./runtime-{num_repeat}_iterations-biovol.txt', 'w')
for i, augmentation in enumerate(augmentations_to_check): # random_scale_transform
for i, augmentation in enumerate(augmentations_to_check):
# augmentation = augmentation_getter(augmentations_to_check, i, size)
aug_name = augmentation.__class__.__name__
print(aug_name)
......@@ -107,13 +95,15 @@ def transformation_speed_benchmark(iterations):
# run for the first time to allocate memory etc.
first_time = time.time()
aug_data = aug(**data)
first_result = time.time() - first_time
first_result = (time.time() - first_time) * 1000 # convert to milliseconds
# run the measured runs
maximum, cumulative = single_transform(iterations, size, augmentation)
result_time = cumulative / iterations
times_arr = single_transform(iterations, size, augmentation)
# log
log_message = f'Runtime in seconds. ' \
f'FirstRun: {first_result:.3f}, Average: {result_time:.3f}, Maximum: {maximum:.3f}. ' \
f'FirstRun: {first_result:.3f}, Average: {times_arr.mean():.3f}, ' \
f'Maximum: {times_arr.max():.3f}, Std: {times_arr.std():.3f}. ' \
f'(Transform: {aug_name}, Iterations: {iterations}, ImageSize: {size})\n'
f.write(log_message)
print(log_message)
......
tests/speed_test_other.py
+ 8
− 14
View file @ f9d3f1b2
......@@ -249,10 +249,8 @@ def transform_data(lib, data, pipeline):
pass
def single_transform(iterations, shape, augmentation, lib):
cumulative = 0
maximum = 0
def single_transform(iterations, shape, augmentation, lib) -> np.ndarray:
times = np.zeros(iterations)
for i in range(iterations):
# prepare data and transformation pipeline
transformation_pipeline = init_compose(lib, augmentation)
......@@ -262,13 +260,9 @@ def single_transform(iterations, shape, augmentation, lib):
t_0 = time.time()
_ = transform_data(lib, data, transformation_pipeline)
time_spent = time.time() - t_0
times[i] = time_spent * 1000 # convert from seconds to milliseconds
# accumulate time
cumulative += time_spent
if time_spent > maximum:
maximum = time_spent
return maximum, cumulative
return times
def transformation_speed_benchmark(iterations):
......@@ -290,15 +284,15 @@ def transformation_speed_benchmark(iterations):
data = get_input_data(lib, shape)
first_time = time.time()
_ = transform_data(lib, data, transformation_pipeline)
first_result = time.time() - first_time
first_result = time.time() - first_time # this is in seconds !!!
# the measured runs
maximum, cumulative = single_transform(iterations, shape, augmentation, lib)
result_time = cumulative / iterations
times_arr = single_transform(iterations, shape, augmentation, lib) # this is in milliseconds
# logging
log_message = f'Runtime in seconds. ' \
f'FirstRun: {first_result:.3f}, Average: {result_time:.3f}, Maximum: {maximum:.3f}. ' \
f'FirstRun: {first_result:.3f}, Average: {times_arr.mean():.3f}, ' \
f'Maximum: {times_arr.max():.3f}, Std: {times_arr.std():.3f}. ' \
f'(Library: {lib}, Transform: {aug_name}, Iterations: {iterations}, ImageSize: {shape})\n'
f.write(log_message)
print(log_message)
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment