LLM Benchmark (LLaMA 7B v2)

Visit this form to submit your benchmark

Rank 🤖 Machine 📊 GPU Cores 🗳️Bandwidth (GB/s) F16 PP (T/S) F16 TG (T/S) ⭐ Q4 PP (T/S) ⭐ Q4 TG (T/S) Price (T/$)
1 M2 Ultra 76 800 1401.85 41.02 1238.48 94.27 2603.04 7.62
2 M2 Ultra 60 800 1128.59 39.86 1013.81 88.64 2953.37 7.53
3 M1 Ultra 64 800 1168.89 37.01 1030.04 83.73 1076.06 6.04
4 M1 Ultra 48 800 875.81 33.92 772.24 74.93 4821.86 5.92
5 M3 Max 40 400 779.17 25.09 759.7 66.31 1803.09 3.59
6 M2 Max 38 400 755.67 24.65 671.31 65.95 2928.87 8.77
7 M1 Max 32 400 599.53 23.03 530.06 61.19 2602.22 8.47
8 M2 Max 30 400 600.46 24.16 537.6 60.99 5161.45 3.52
9 M3 Max 30 300 589.41 19.54 567.59 56.58 49.04 1.12
10 M1 Max 24 400 453.03 22.55 400.26 54.61 2219.27 3.56
11 M2 Pro 19 200 384.38 13.06 341.19 38.86 5088.19 7.49
12 M2 Pro 16 200 312.65 12.47 294.24 37.87 4789.04 1.50
13 M1 Pro 16 200 302.14 12.75 266.25 36.41 1878.63 9.23
14 M1 Pro 14 200 262.65 12.75 232.55 35.52 5185.03 3.60
15 M3 Pro 18 150 357.45 9.89 341.67 30.74 3158.03 0.14
16 M3 Pro 14 150 null null 269.49 30.65 3757.88 5.45
17 M2 10 100 201.34 6.72 179.57 21.91 3825.26 9.41
18 M2 8 100 null null 145.91 21.7 2199.20 8.89
19 M3 10 100 null null 186.75 21.34 2268.93 0.38
20 M1 7 68 null null 107.81 14.19 4381.81 7.18
21 M1 8 68 null null 117.96 14.15 2508.85 8.71
22 M3 8 100 null null null null 2830.09 9.86
23 M3 Ultra 60 800 null null null null 1385.51 5.27
24 M3 Ultra 80 800 null null null null 3000.49 5.49

Sources: