Adog64/Adversarial-Machine-Learning-Clinic
1
0
Fork 0
Code Issues 3 Pull Requests Packages Projects 1 Releases Wiki Activity

1-Bit, color snapping, and pluality vote accuracies

Browse Source
This commit is contained in:
Aidan Sharpe
2024-04-07 12:04:46 -04:00
parent 30d8766e69
commit 8f264f7958
3 changed files with 134 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
Filter_Analysis/wiki/Results.md
View File

@ -46,64 +46,87 @@
## Models Defended with Various Filters
### Tabulated Results
| $\epsilon$ | FGSM | Kuwahara | Bilateral | Gaussian Blur | Random Noise | Snapped Color |
|------------|--------|----------|-----------|---------------|--------------|---------------|
| 0.05 | 0.9600 | 0.8700 | 0.8902 | 0.9271 | 0.9603 | 0.9781 |
| 0.10 | 0.8753 | 0.8123 | 0.8133 | 0.8516 | 0.8677 | 0.8818 |
| 0.15 | 0.7229 | 0.7328 | 0.7098 | 0.7415 | 0.7153 | 0.8408 |
| 0.20 | 0.5008 | 0.6301 | 0.5683 | 0.5983 | 0.4941 | 0.7496 |
| 0.25 | 0.2922 | 0.5197 | 0.4381 | 0.4591 | 0.2843 | 0.4301 |
| 0.30 | 0.1599 | 0.3981 | 0.3364 | 0.3481 | 0.1584 | 0.2091 |
| $\epsilon$ | Unfiltered | Kuwahara | Bilateral | Gaussian Blur | Random Noise | Snapped Color | 1-Bit | Plurality |
|------------|------------|----------|-----------|---------------|--------------|---------------|--------|-----------|
| 0.00 | 0.992 | 0.9066 | 0.9391 | 0.9682 | 0.9911 | 0.9913 | 0.9722 | 0.9889 |
| 0.05 | 0.9600 | 0.8700 | 0.8902 | 0.9271 | 0.9603 | 0.9781 | 0.8409 | 0.9600 |
| 0.10 | 0.8753 | 0.8123 | 0.8133 | 0.8516 | 0.8677 | 0.8818 | 0.7919 | 0.8799 |
| 0.15 | 0.7229 | 0.7328 | 0.7098 | 0.7415 | 0.7153 | 0.8408 | 0.7329 | 0.7879 |
| 0.20 | 0.5008 | 0.6301 | 0.5683 | 0.5983 | 0.4941 | 0.7496 | 0.6794 | 0.6467 |
| 0.25 | 0.2922 | 0.5197 | 0.4381 | 0.4591 | 0.2843 | 0.4301 | 0.6233 | 0.4721 |
| 0.30 | 0.1599 | 0.3981 | 0.3364 | 0.3481 | 0.1584 | 0.2091 | 0. | |
### Plotted Results
![]()
![Results Plot](../Plurality_Vote_Accuracy_Agaisnt_Individual_Filters.png)
### Raw Program Output
Epsilon: 0.05
====== EPSILON: 0.0 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 9600 / 10000 = 0.96
Attacked (Kuwahara Filter) Accuracy = 8700 / 10000 = 0.87
Attacked (Bilateral Filter) Accuracy = 8902 / 10000 = 0.8902
Attacked (Gaussian Blur) Accuracy = 9271 / 10000 = 0.9271
Attacked (Random Noise) Accuracy = 9603 / 10000 = 0.9603
Attacked (Snapped Color) Accuracy = 9781 / 10000 = 0.9781
Epsilon: 0.1
Unfiltered Accuracy = 9920 / 10000 = 0.992
Kuwahara Filter Accuracy = 9066 / 10000 = 0.9066
Bilateral Filter Accuracy = 9391 / 10000 = 0.9391
Gaussian Blur Accuracy = 9682 / 10000 = 0.9682
Random Noise Accuracy = 9911 / 10000 = 0.9911
Snapped Color Accuracy = 9913 / 10000 = 0.9913
1 Bit Accuracy = 9722 / 10000 = 0.9722
Plurality Vote Accuracy = 9889 / 10000 = 0.9889
====== EPSILON: 0.05 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 8753 / 10000 = 0.8753
Attacked (Kuwahara Filter) Accuracy = 8123 / 10000 = 0.8123
Attacked (Bilateral Filter) Accuracy = 8133 / 10000 = 0.8133
Attacked (Gaussian Blur) Accuracy = 8516 / 10000 = 0.8516
Attacked (Random Noise) Accuracy = 8677 / 10000 = 0.8677
Attacked (Snapped Color) Accuracy = 8818 / 10000 = 0.8818
Epsilon: 0.15000000000000002
Unfiltered Accuracy = 9600 / 10000 = 0.96
Kuwahara Filter Accuracy = 8700 / 10000 = 0.87
Bilateral Filter Accuracy = 8902 / 10000 = 0.8902
Gaussian Blur Accuracy = 9271 / 10000 = 0.9271
Random Noise Accuracy = 9587 / 10000 = 0.9587
Snapped Color Accuracy = 9781 / 10000 = 0.9781
1 Bit Accuracy = 8409 / 10000 = 0.8409
Plurality Vote Accuracy = 9600 / 10000 = 0.96
====== EPSILON: 0.1 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 7229 / 10000 = 0.7229
Attacked (Kuwahara Filter) Accuracy = 7328 / 10000 = 0.7328
Attacked (Bilateral Filter) Accuracy = 7098 / 10000 = 0.7098
Attacked (Gaussian Blur) Accuracy = 7415 / 10000 = 0.7415
Attacked (Random Noise) Accuracy = 7153 / 10000 = 0.7153
Attacked (Snapped Color) Accuracy = 8408 / 10000 = 0.8408
Epsilon: 0.2
Unfiltered Accuracy = 8753 / 10000 = 0.8753
Kuwahara Filter Accuracy = 8123 / 10000 = 0.8123
Bilateral Filter Accuracy = 8133 / 10000 = 0.8133
Gaussian Blur Accuracy = 8516 / 10000 = 0.8516
Random Noise Accuracy = 8696 / 10000 = 0.8696
Snapped Color Accuracy = 8818 / 10000 = 0.8818
1 Bit Accuracy = 7919 / 10000 = 0.7919
Plurality Vote Accuracy = 8799 / 10000 = 0.8799
====== EPSILON: 0.15000000000000002 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 5008 / 10000 = 0.5008
Attacked (Kuwahara Filter) Accuracy = 6301 / 10000 = 0.6301
Attacked (Bilateral Filter) Accuracy = 5683 / 10000 = 0.5683
Attacked (Gaussian Blur) Accuracy = 5983 / 10000 = 0.5983
Attacked (Random Noise) Accuracy = 4941 / 10000 = 0.4941
Attacked (Snapped Color) Accuracy = 7496 / 10000 = 0.7496
Epsilon: 0.25
Unfiltered Accuracy = 7229 / 10000 = 0.7229
Kuwahara Filter Accuracy = 7328 / 10000 = 0.7328
Bilateral Filter Accuracy = 7098 / 10000 = 0.7098
Gaussian Blur Accuracy = 7415 / 10000 = 0.7415
Random Noise Accuracy = 7119 / 10000 = 0.7119
Snapped Color Accuracy = 8408 / 10000 = 0.8408
1 Bit Accuracy = 7329 / 10000 = 0.7329
Plurality Vote Accuracy = 7879 / 10000 = 0.7879
====== EPSILON: 0.2 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 2922 / 10000 = 0.2922
Attacked (Kuwahara Filter) Accuracy = 5197 / 10000 = 0.5197
Attacked (Bilateral Filter) Accuracy = 4381 / 10000 = 0.4381
Attacked (Gaussian Blur) Accuracy = 4591 / 10000 = 0.4591
Attacked (Random Noise) Accuracy = 2843 / 10000 = 0.2843
Attacked (Snapped Color) Accuracy = 4301 / 10000 = 0.4301
Epsilon: 0.3
Unfiltered Accuracy = 5008 / 10000 = 0.5008
Kuwahara Filter Accuracy = 6301 / 10000 = 0.6301
Bilateral Filter Accuracy = 5683 / 10000 = 0.5683
Gaussian Blur Accuracy = 5983 / 10000 = 0.5983
Random Noise Accuracy = 4933 / 10000 = 0.4933
Snapped Color Accuracy = 7496 / 10000 = 0.7496
1 Bit Accuracy = 6794 / 10000 = 0.6794
Plurality Vote Accuracy = 6467 / 10000 = 0.6467
====== EPSILON: 0.25 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Attacked (No Filter) Accuracy = 1599 / 10000 = 0.1599
Attacked (Kuwahara Filter) Accuracy = 3981 / 10000 = 0.3981
Attacked (Bilateral Filter) Accuracy = 3364 / 10000 = 0.3364
Attacked (Gaussian Blur) Accuracy = 3481 / 10000 = 0.3481
Attacked (Random Noise) Accuracy = 1584 / 10000 = 0.1584
Attacked (Snapped Color) Accuracy = 2091 / 10000 = 0.2091
Unfiltered Accuracy = 2922 / 10000 = 0.2922
Kuwahara Filter Accuracy = 5197 / 10000 = 0.5197
Bilateral Filter Accuracy = 4381 / 10000 = 0.4381
Gaussian Blur Accuracy = 4591 / 10000 = 0.4591
Random Noise Accuracy = 2876 / 10000 = 0.2876
Snapped Color Accuracy = 4301 / 10000 = 0.4301
1 Bit Accuracy = 6233 / 10000 = 0.6233
Plurality Vote Accuracy = 4721 / 10000 = 0.4721
====== EPSILON: 0.30000000000000004 ======
Clean (No Filter) Accuracy = 9920 / 10000 = 0.992
Unfiltered Accuracy = 1599 / 10000 = 0.1599
Kuwahara Filter Accuracy = 3981 / 10000 = 0.3981
Bilateral Filter Accuracy = 3364 / 10000 = 0.3364
Gaussian Blur Accuracy = 3481 / 10000 = 0.3481
Random Noise Accuracy = 1560 / 10000 = 0.156
Snapped Color Accuracy = 2091 / 10000 = 0.2091
1 Bit Accuracy = 5462 / 10000 = 0.5462
Plurality Vote Accuracy = 3312 / 10000 = 0.3312