2 vs 20 minutes is a big jump - but the XP2 RAW files are quite big... around 16MB bigger than X-T1 (48+ vs 32+).
Using a UHS-II card reader? I'll assume you are, but that would be one bottle neck if you weren't.
How many pictures are you transferring? That of course would be a pretty big variable to note - or are you saying 20 minutes for a single image?
The rest would be around harddrive speed, whether or not your USB drivers are up to date, other things writing to the HD during the import, etc.
There are a lot of variable involved: Card speed, number and size of files, HD Speed, USB speed and/or bottlenecks (a lot of things using the USB Bus), card reader type, and sometimes it can be something finicky like the USB driver (windows 10 did this to me with certain USB ports: 'Your device could operate faster if you plugged into a 3.0 port' etc - despite being plugged in to a USB 3.0 port). I had to upgrade drivers to improve things when I noticed a little slowness.
You could do a control of maybe 10-20 pics and try to isolate the timing, and repeat copies to see what else spikes while you copy (in case any anti-virus or other software spins up - I doubt it, but like to consider everything, since I suspect most AV programs likely won't recognize what a RAF file is).
Edit: Re-read the original post. If you think the files coming form the xP2 might somehow be suspect - take 500MB of pics with same card on X-t1 and transfer and time, then take 500MB with the xpro2 and do the same. Results should match pretty closely.