There is no known limit to the size of the catalogue which is just a database. Biggest one I have read about to date was for just over 13 million files.
That's just it, as a retired DBA, I know there is a difference between "Being able to run" and "runs well".
In a relational database, which I believe the Photos database is, 13 million photos, are more than 13 million files. there are indexes, Album records, collection records, at least one Library record (possibly more. the user will likely break a database that large into multiple Libraries, each with it's own overhead, for response time reasons.). Even a hierarchical database, while faster than a Relational DB, for huge data stores, will not be as fast as it would on a much smaller Data Store, depending on the hardware on which it runs.
Now, if you try to run something that large on, say, a Mac Mini, you will run into problems. Add into the real issue that, Photos, as a free app, is not going to get priority attention when it comes to maintenance, fixing Errors & Omissions, that sort of thing. I won't get into finding a specific photo with collections and albums only. If you don't know exactly WHICH of those the photo is likely to be in. (Gonna be SLOW)
This is why I would say the Photos app should be left to the casual user, and there's nothing wrong with that. It is perfect for my wife, for instance. It's easy to use, and better than Google photo software, (She doesn't even know she is doing backups on her Mac mini. She's happy, and I'll look like a hero at some point in the future!
But I just can not recommend Photos for ANY photo database containing images that will cause some sort of financial, or business loss .
A database that large will have a lot of economic value to a photographer or an organization. Shoot, my 20,000 photo database has economic value, to me, at least. So high speed back ups and disaster recovery, all become issues for that large database. Simple backup utilities built into the OS, like TimeMachine are not up to servicing that sort of database. A backup would take days and days, could actually be a week or more.
I might trust a backup system run by Lightroom, ACDSee, On1, etc. for my puny photo library, but NOT for a 13 million photo database. THAT database is irreplaceable, losing my database would cost some money maybe, but it would not turn my life upside down. So I trust the backups from the dedicated apps as adequate for My needs.
I like Photos, for what it is. But it is not the tool for everyone.