There are several sub-devices, each having its own speed, that can limit your download time.
The line is
SD-Card -> in-camera USB-module -> computer USB-module.
And if you have bottleneck in one of these steps - you limit the whole system.
USB 2.0 on my ancient Canon 350D gave 3 Mbytes/sec downloading speed.
"Same" USB 2.0 on Oly Pen-1 or Canon 5Dmk2 gives 6-7 Mbytes/sec.
"Same" USB 2.0 on my Nikon D600 gives 25-26 Mbytes/sec.
"Same" USB 2.0 on my external HDD on an old PC gave 20-22 Mbytes/sec.
"Same" USB 2.0 on my external HDD connected to the good PC gives 30 (not sure about 40) Mbytes/sec reading speed.
Personally I would better have top USB 2.0 than cheapest USB 3.0 crap.
having USB 3.0 alone does not guarantee you anything. If your card is rated 40 mMbit/sec for reading, than 5 Mbyte/sec is best you can get, independently on what USB you have.