TL;DR: Buy the system that gives you the native lenses you want. My personal opinion is Sony's cameras are far better than Fuji's, but, Sony doesn't give me the lenses I want, no other system with a large enough (for my preference) sensor does, not natively to the mount and format anyway, so I put up with what I consider to be outdated, poorly implemented camera bodies because in the end, Fuji JPEG + Fuji glass delivers.
I shot about 20-30k images on both systems over the last few years. Most recently I was using an A7 II and switched to Fuji. It was sort of an accident. You see, I was getting kind of frustrated that all the lenses available natively for the A7 system were big or smaller aperture than I wanted or just not a focal length I liked. I don't mind some lenses being big. I don't choose mirrorless for the size which I think is a farce. I choose it mainly for the high quality EVF style of shooting (and by EVF I also include LCD, point is the sensor is providing the image by which I compose my shots.) and simpler (thus longer lasting) design.
Anyway, I didn't do a ton of photography this last year or so due to time and expanding interests in other areas. So, I decided to sell off all my A7 II gear and buy a Sony RX1R II. However, during November while I was waiting for that camera to be released and my pre-order fulfilled, I got cold feet on spending $3300 on a fixed lens camera and canceled the pre-order. Xmas was coming up and I always end up shooting a number of family type situations that are important to me so I needed a good quality camera. I decided to come back to Fuji, mainly because it has a 23/1.4 which is 35/2 equivalent, same as the RX1 series, and a lens I sorely miss from the A7 kit. I was also attracted by the existence of a general purpose zoom that is not huge (18-135) and an excellent TRUE portrait length prime (90/2) as well as a few other favorites (14mm f/2.8, Rokinon 8mm fisheye, and then upcoming 100-400mm zoom with teleconverter.)
I grabbed an X-T10 and a few of the lenses including the 35/1.4 because obviously, gotta have that. And have been pretty happy for the most part. It should be obvious that the reason I ended up going with Fuji instead of back to A7 II or A7R II is because of the glass options available in native form. (Despite what people say, adapted lenses on A7 cameras don't AF nearly as fast as native ones do. It's nice to have the option but I would never buy a system based on the premise of third party adapted lenses.)
That said, I think the Fuji cameras themselves really suck compared to the competition. I hate the retro handling. I hate how bad they are at EVF implementations (For example, Sony cameras NEVER stutter, and I'm not talking about exposure time blackout, obviously this happens, I'm talking about the stutter that happens on Fuji when autofocusing and doing other similar things.) I hate how limited the autofocus system is, even after the updates, especially compared to the very latest Sony bodies with dynamic AF point resizing and such. Sony's body performance is slick as butter. Best you can get without dropping to comically small sensor sizes. So I really miss that. I put up with sub-par body design to get at the lenses I want.
Fuji always promises cool features and occasionally delivers. The huge EVF in the X-T1, for example, is beautiful, and at least on that one camera, the stuttering is less noticeable, I assume due to having a faster processor. But frankly, even Fuji's latest "X Processor Pro" is built on 4 years out of date ARM technology and it's not even operating fully on the X-Pro2. They are very behind in this area.
Other things that I miss from Sony are the near silent lenses that AF super fast without hunting all around like a DSLR in live view mode. A few of Fuji's lenses have more modern design here, but, ALL of the FE lenses have as good or better motor implementations. Their latest piezo electric designs are shockingly fast and quiet.
Also every Sony FE lens is weather sealed. Every single one. Same with the (full frame) camera bodies.
But, the cost is high, and the lens options I desire do not exist. I mean, a few do. The Sony FE 55mm f/1.8 is easily the most satisfying lens I've ever used in terms of image quality in every aspect. I miss that lens the most. But honestly I don't miss any other lens from that system. Now they are about to release some masterful zooms and portrait prime, which I'm sure will be nice but I don't really like normal range zooms, and the 70-200/2.8 doesn't even have a price. Plus it's big. Not really a lens class I tend to prefer even on DSLR. Furthermore, 35mm always ends up my preferred general use focal length and the Sony-Zeiss 35mm f/1.4 is overpriced for its performance and quite large while the 35mm f/2.8 is a great lens, and super portable, it's only f/2.8. f/2 is the smallest effective aperture I can use to get the results I want.
Almost all of this is a matter of my personal preference and expectations. I can and have made some of my all time most favorite images on Fuji cameras. I print SOOC more often with Fuji than any other system I've shot (APS-C & FF Nikon, Panasonic MFT, APS-C and FF Sony E mount, and a number of advanced compacts.) So they get the job done, it just doesn't have the fit and finish when compared directly to other offerings. In the back of my mind, I always know what I'm missing.
Honestly, for me, Fuji is appealing because of 1) lens options that I want and 2) the quality of the JPEGs at normal (18x12 to 24x18) print sizes. And in the end, what is the most important thing? Also, I like how hard Fuji seems to be trying. They are kind of an underdog, learning as they go, and I believe they have a chance at catching up someday, but things like the X-Pro2 being basically on par with Sony's years-old a6300 in most respects is disheartening. They need to jump ahead. I feel like X-T2 is their last chance to do this or be disregarded indefinitely. Sony is actually producing new lenses VERY fast and finally doing some true bare knuckle lens design R&D. Their XA element approach solves the biggest problem in modern lens design today: onion bokeh from aspherical elements. That's improvement with substance.
Right now I'm considering a mixed system, using Fuji cameras where they give the biggest advantage, mainly in long lens shooting as well as a few of the more obscure focal lengths that I enjoy (that Rokinon 8mm fisheye for example is very special, with a unique projection and the latest version is very well corrected.) And grabbing an RX1R to replace the shooting I end up doing with the 23/1.4 and 35/1.4 lenses.
I'd end up with an X-T10 (until the X-T2 comes), 8mm fish, 14/2.8, 90/2, 18-135 and 100-400 + TC and the RX1R II and know that I'll shoot with the RX1R II most often as my go-to. But, it's not decided. Having a mixed system can be a little hard, as I've tried it before, because you end up sort of wishing one camera was the other and sometimes aren't sure where to invest. At least in this setup I wouldn't have to wonder if I should buy any lenses for the RX1R II, but, it may also constantly make me think about selling off the Fuji gear and just going with an A7R II and living with the more limited lens options for the time being. Tough decisions, and a lot of money at stake.