GPUs are a SIMD architecture designed for 4x4 float32 matrix multiplications found in video games.
But NPUs (like Googles NPU) are systolic arrays designed for 16x16 or even 256x256 float16 or even int8 matrix multiplications instead.
-------
NVidia builds larger matrix multiplications out of the 4x4 float16 base that a SM is designed for. After all, a 8x8 matrix multiplication is just four of the smaller 4x4 matrix multiplications.
Yes, it's eight multiplication with the naive algorithm. You can get to seven with a clever trick (Strassen's algorithm). As far as I'm aware, with these precise numbers (8x8 matrix decomposed into four 4x4) it's unknown if we can do better.
But NPUs (like Googles NPU) are systolic arrays designed for 16x16 or even 256x256 float16 or even int8 matrix multiplications instead.
-------
NVidia builds larger matrix multiplications out of the 4x4 float16 base that a SM is designed for. After all, a 8x8 matrix multiplication is just four of the smaller 4x4 matrix multiplications.