Just to clarify USB 2.0 is limited to about 23 MB/sec in practice and USB 3.0 to more like 150-200 MB/sec.
A typical HDD can shift data at about 80-150 MB/sec ( varies over the position on the drive as well as the specific model ).
A typical SDD is very fast in two ways :
(1) Transferring large amounts of data serially ( e.g. GB's at a go ), where they can reach as high as 500 MB/sec ( which you only get if you use an internal drive and only then on SATA III and in ideal conditions ). SATA II interfaces are limited to 300 MB/sec even in ideal conditions.
(2) Randomly accessing data all over the drive ( a typical boot up or server behavior ). This is why SDDs are popular for speeding up boot-up and application launch and for servers. It can make a desktop seem a bit more responsive.
In practical terms using an SDD should be internally as your main drive for booting and storing applications. Data can be stored on larger and cheaper HDDs with almost no practical impact on performance.
An external HDD over a USB 3.0 connection is very fast in practice.