LR Catalog Question

I think I goofed by puting all my images from three drives into a single LR3 catalog.

I have 30,000 images so far, and my "Photos 3" drive has almost 1 TB of space left for additional images! Now I've read that LR3 will slow down beyond 50,000 photos or so, and I'll surely exceed that before I fill my third drive.

I'd like to make a separate catalog for the almost-empty 1TB drive, but I first I have to remove that drive's folders from the present catalog. This is where I get stuck.

I see LR provisions for removing files or folders from a catalog, but not entire drives. I suppose I can remove the folders (about 30) one at a time, but will I be left with an empty "Photos 3" drive showing in my "Folders" panel?

Is there a better way to remove the entire 3rd drive from the catalog in a single operation? Any caveats in this operation?

Thanks again!
--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

JerryG1 said:

Now I've read that LR3 will slow down beyond 50,000 photos or so, and I'll surely exceed that before I fill my third drive.

Click to expand...

I'm sure that's not true. I have way over 50,000 and have heard people with hundreds of thousands of images say there's no significant performance penalty. I'm pretty sure Adobe have said the catalogue is designed to scale to very large numbers of images, though I've can't give you a reference.

The whole point of the catalogue is that all your photos are in one database. That's what it's designed for.

--
Simon

I have 82,600 in mine, on a not new computer, using external drives, and the speed is OK for me. Even search across the entire catalog. I'm using LR 3.5.

I prefer my images on a single drive and then backup to another drive. There are lots of backup programs that will keep them in sync. I keep my backup drive turned off except when I'm backing up, which I start manually.

Moving things. I always have to play around a bit -- I've been with LR since beta and things are always changing -- making it easier if you know the new ways. I "move" the files myself and then repoint LR because moving within LR is slower. I always have a copy - I don't MOVE anything, just copy, rename the old so LR can't find it, repoint LR, then finally delete after a copy of the new location is made. Yes, I'm paranoid.

It's easiest if you have all of your images in a single directory structure so you can repoint everything with one click. If you don't, there is now a way to get an upper directory showing in LR but I forget how right now. At worst, you could put an image to that umbrella directory and import it, but I don't think that's necessary any more.

Once you know what you want to do exactly, you might post again and ask a more specific question.

--
Judy
http://nichollsphoto.com/

Thanks for responding.

I certainly won't argue that I'm asking for something sensible. I'm brand new to LR (vers 3.5), have Scott Kelby's book in my left hand and am typing with my right. Scott discusses avoiding slow downs via multiple catalogs. I know I read someplace that it becomes noticable at 50-60 thousand images, but can't find that reference now.

Anyway I'm glad to know that neither of you are experiencing slowdowns with many more images than I have. Still, I would like to know how to remove entire drives from a catalog since my catalog is set up to cover three drives. I currently have three drives for photos, and may not always.
--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

JerryG1 said:

I certainly won't argue that I'm asking for something sensible. I'm brand new to LR (vers 3.5), have Scott Kelby's book in my left hand and am typing with my right. Scott discusses avoiding slow downs via multiple catalogs. I know I read someplace that it becomes noticable at 50-60 thousand images, but can't find that reference now.

Click to expand...

Well I'm not an expert and Scott Kelby is! However, I've read a few comments (here and on http://forums.adobe.com/community/lightroom ) that on this one respect the Scott Kelby view of having multiple catalogues (partly for performance reasons) is not generally shared by other experts. But that's hearsay - I haven't done extensive tests.

--
Simon

JerryG1 said:

Still, I would like to know how to remove entire drives from a catalog since my catalog is set up to cover three drives. I currently have three drives for photos, and may not always.

Click to expand...

It makes no basic difference to LR what volume an image is on, what folder it is in, and what it is called. These are all just accidental properties of the file - some neutral data that make zero difference to the things that LR does with it (or, better, that YOU do to and with it inside LR).

[Each image is either online (found - fully available) or offline (can't find or connect to where the image was previously, so certain operations are temporarily unavailable) at a given moment. However LR's information about all of those images, online or offline, and no matter on what volume - plus a stored preview for each - are all located in the same single place.]

If you did want to make a different catalog holding some of the images that are currently shown in your main catalog, you can highlight them using whatever method you like inside your Library view, and then Export As Catalog. This is all explained on the AdobeTV site videos and elsewhere. The resulting new Catalog holds copies of all the same editing work etc, as your main one does. It may refer to (share) the same image files or else duplicate copies of those may be made as well, depending on the option you pick. Then (if desired) these same selected images can be removed from your main Catalog - carefully.

However I would agree with others that unless and until you are experiencing some kind of substantial difficulty, the single Catalog makes more logical and practical sense. Working over multiple volumes is no problem. Also images can be moved around... again, the mechanics of this need a little explanation and awareness since these are not simple unattached files, but files that are linked to each at a given address - an address that will also need to be updated to suit. However, it is certainly not hard to do... this kind of situation is the very job that LR exists to manage.

RP

richardplondon said:

JerryG1 said:

Still, I would like to know how to remove entire drives from a catalog since my catalog is set up to cover three drives. I currently have three drives for photos, and may not always.

Click to expand...

...

If you did want to make a different catalog holding some of the images that are currently shown in your main catalog, you can highlight them using whatever method you like inside your Library view, and then Export As Catalog. This is all explained on the AdobeTV site videos and elsewhere.

Click to expand...

Thanks for reminding me about Adobe TV. I saw a couple of videos a week or so ago, but carefully avoided anything that sounded as esoteric as "Exporting Catalogs." I'll take a look.

richardplondon said:

However I would agree with others that unless and until you are experiencing some kind of substantial difficulty, the single Catalog makes more logical and practical sense.

Click to expand...

I see the search advantage of a single catalog. I was concerned about future problems, and feared that the longer I waited the harder the fix would be. (I know a wedding photgrapher, shoots a couple of thousand frames a week, and gave up on LR because it "bogged down" after a while. Don't know details.)

Since I have no LR problems now, I'll learn some more before deciding whether multiple catalogs makes sense for me.

--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

I also recommend maintaining a single catalog for the sake of keeping things "simple". I have 93,000 images in my catalog. LR performance is excellent although I will admit I built my own new PC sevral months ago(good father & son project). (Intel Sandybridge I5-2600 CPU, 8gb ram, LR's catalog, previews, and my image library reside on internal 2TB mirrored drive.)

regards, Art

Miceli Photography said:

I also recommend maintaining a single catalog for the sake of keeping things "simple". I have 93,000 images in my catalog. LR performance is excellent although I will admit I built my own new PC sevral months ago(good father & son project). (Intel Sandybridge I5-2600 CPU, 8gb ram, LR's catalog, previews, and my image library reside on internal 2TB mirrored drive.)

regards, Art

Click to expand...

I've decided to stay with the single catalog, unless and until I see a reason to change.

About your new PC, Do you have a third internal drive for OS and others, or is that on the two mirrored drives?

I'm interested in a RAID system--getting tired of dealing with backups.

--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

In my experience LR does slow down after you have imported several thousands of photographs...but not all the time. It can go through phases where it is fine and then other times it appears to be doing something in the background..writing xmp files etc and it can grind to a halt for a while.

I do have one master catalogue with everything in it. However, I also have a "working catalogue" which is empty except for my latest shoot of a few hundred photos. I do my editing and keywording etc. in the working catalogue and when I am done I import these settings into my master catalogue.

You need to make sure that the settings for both catalogues are the same and that the keywords and structure are identical. This often has to be one manually but if you regularly maintain it is not an issue.

JerryG1 said:

I'm interested in a RAID system--getting tired of dealing with backups.

Click to expand...

With RAID you don't have all your eggs in one basket. You can spread them over two baskets, but you are still carrying them around, one in each hand . If you trip over, get ready to eat a great deal of omelette.

RAID is great, especially sometimes in performance terms. But it gives no added protection against most of the bad things that happen to computers - theft, fire, building collapse, flooding. Pilot error (deleting or messing up the data in some way that is not immediately detected). Data content corruption. Asteroid strike . Failure of the RAID controller (an added risk, though rare) may leave you with unreadable array disks.

I am sure I am preaching to the choir, but painful experience says that EVERYONE needs some form of redundant backups (plural) however occasional - however reliable their main system; and that these need not to participate in the same risks that their main system is subject to. They also need to spread back in time a bit - rather than merely mirroring live data. That's what troubles me about online systems: some of the data you have been syncing may turn out to have been junk for the last little while.

The trick is IMO, to firstly make a cycling or accumulating backup routine as convenient as you can, and then to tie it into some repeating habit while you are still fortunate enough not to need to call on it.

RP

MarkNicholas said:

In my experience LR does slow down after you have imported several thousands of photographs...but not all the time. It can go through phases where it is fine and then other times it appears to be doing something in the background..writing xmp files etc and it can grind to a halt for a while.

Click to expand...

How many thousands?

I have about 30,000 images imported now. Normally, the "grid View" fills with "All Photographs" faster than I can scroll down--seems like just a couple of seconds.

The one time I noticed a serious slow-down was when I accidentally used a back-up catalog from a USB connected external drive. Easily fixed!

From your description of the problem, I agree you probably have some background operation going on --may not be LR-- when you see the slow-down.

I use a two year old I5, 8GB, Windows 7 machine.

Hopefully, I won't have any problems for a couple of years. By then we'll probably have moved on to a version (and computer) that can handle even bigger databases.

--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

JerryG1 said:

MarkNicholas said:

In my experience LR does slow down after you have imported several thousands of photographs...but not all the time. It can go through phases where it is fine and then other times it appears to be doing something in the background..writing xmp files etc and it can grind to a halt for a while.

Click to expand...

How many thousands?

I have about 30,000 images imported now. Normally, the "grid View" fills with "All Photographs" faster than I can scroll down--seems like just a couple of seconds.

Click to expand...

No, I don't get any slow-down either with > 50,000 images.

JerryG1 said:

JerryG1 said:

I do have one master catalogue with everything in it. However, I also have a "working catalogue" which is empty except for my latest shoot of a few hundred photos. I do my editing and keywording etc. in the working catalogue and when I am done I import these settings into my master catalogue.

You need to make sure that the settings for both catalogues are the same and that the keywords and structure are identical. This often has to be one manually but if you regularly maintain it is not an issue.

Click to expand...

Click to expand...

Coor, that sounds complicated!

From my experience, I would get no performance benefit by doing that (as opposed to importing directly into my one-and-only catalogue).

--
Simon

Simon Garrett said:

JerryG1 said:

MarkNicholas said:

In my experience LR does slow down after you have imported several thousands of photographs...but not all the time. It can go through phases where it is fine and then other times it appears to be doing something in the background..writing xmp files etc and it can grind to a halt for a while.

Click to expand...

How many thousands?

I have about 30,000 images imported now. Normally, the "grid View" fills with "All Photographs" faster than I can scroll down--seems like just a couple of seconds.

Click to expand...

No, I don't get any slow-down either with > 50,000 images.

Simon Garrett said:

Simon Garrett said:

I do have one master catalogue with everything in it. However, I also have a "working catalogue" which is empty except for my latest shoot of a few hundred photos. I do my editing and keywording etc. in the working catalogue and when I am done I import these settings into my master catalogue.

You need to make sure that the settings for both catalogues are the same and that the keywords and structure are identical. This often has to be one manually but if you regularly maintain it is not an issue.

Click to expand...

Click to expand...

Coor, that sounds complicated!

From my experience, I would get no performance benefit by doing that (as opposed to importing directly into my one-and-only catalogue).

--
Simon

Click to expand...

Actually its not that complicated and I know of several LR gurus on the LR forum who use this method.

I do have a rather ancient system and that probably accounts for my intermittent slow downs. I should say that even with my ancient system it is usually fine for most of the time.

You should try firing up a catalogue with 50K+ RAW files and one with just 100 RAW files. You will definitely notice the difference

To put it simply, Windows 7 and all the installed programs are on a single "C" drive. My content(Images, music, LR catalog, etc...) are all on the RAID'd drives. As others have since responded, this is NOT a replacement for a robust redundent backup strategy including offsite storage of a backup set; all of which I also employ. (More specifically, disaster recovery(i.e. bare metal restore) backups of the "C" drive and conventional full backups of my content drive(s). Email me and I can share more details.

regards, Art

JerryG1 said:

Miceli Photography said:

I also recommend maintaining a single catalog for the sake of keeping things "simple". I have 93,000 images in my catalog. LR performance is excellent although I will admit I built my own new PC sevral months ago(good father & son project). (Intel Sandybridge I5-2600 CPU, 8gb ram, LR's catalog, previews, and my image library reside on internal 2TB mirrored drive.)

regards, Art

Click to expand...

I've decided to stay with the single catalog, unless and until I see a reason to change.

About your new PC, Do you have a third internal drive for OS and others, or is that on the two mirrored drives?

I'm interested in a RAID system--getting tired of dealing with backups.

--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

Click to expand...

MarkNicholas said:

...

You should try firing up a catalogue with 50K+ RAW files and one with just 100 RAW files. You will definitely notice the difference

Click to expand...

I've no doubt you are right: more data takes longer to process than less data. Whether or not delays at 50,000+ images are personally bothersome depends on individual temprement and the age (speed) of the computer doing the processing.
--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

richardplondon said:

JerryG1 said:

I'm interested in a RAID system--getting tired of dealing with backups.

Click to expand...

With RAID you don't have all your eggs in one basket. You can spread them over two baskets, but you are still carrying them around, one in each hand . If you trip over, get ready to eat a great deal of omelette.
...

Click to expand...

I'm sure you are right, Richard, but I am guilty of not avoiding the risks associated with co-located originals and backups. Also, like many I'm sure, I don't do the redundant thing. A RAID system would pretty much get me the (incomplete) benefits of my external backup drives without having to run backups.

--
JerryG

My galleries at:
http://www.pbase.com/jerryg1

Even with a good computer LR experiences a loss in performance sooner or later. I use a Mac Pro tower and have seen the difference as more photos are added.

How often you should create new catalogs depends on how much you shoot. A somewhat popular method is to create a new catalog every year. I highly recommend this method if you shoot 50,000 or more images in a single year.

If you shoot 10,000 or less images per year, then you can wait a while. But there is no harm in creating numerous catalogs. One catalog per year for 10,000 or less images works fine and helps with organization.

MarkNicholas said:

I do have a rather ancient system and that probably accounts for my intermittent slow downs. I should say that even with my ancient system it is usually fine for most of the time.

Click to expand...

My system is about a year old with an i7-930 processor. Not quite state-of-the-art, but fairly modern.

MarkNicholas said:

You should try firing up a catalogue with 50K+ RAW files and one with just 100 RAW files. You will definitely notice the difference

Click to expand...

I just tried with a catalogue of 86 files, and the speed appeared to be the same as with my 50k+ catalogue!

I'm not sure exactly what to test, but opening LR took (in each case) 4 seconds before it had finished rendering the images in the currently viewed folder. Navigating through the folder, switching between library and develop, all appeared to take the same time.
--
Simon

Z_fan said:

Even with a good computer LR experiences a loss in performance sooner or later. I use a Mac Pro tower and have seen the difference as more photos are added.

Click to expand...

I also have noticed a lot of changes in LR's performance over the years. I use a single Catalog almost exclusively, but sometimes small separate Catalogs for certain isolated projects. Things are probably not quite so snappy as when I first brought my master catalog into this computer when it was new; and there were indeed fewer pictures in it back then.

On the other hand, I am using a camera with much higher resolution now. I used to disable luminance NR altogether, and now LR does excellent NR for me dynamically combined with local adjustments, profiled lens correction and perspective adjustment etc - things that once would have all been done externally; LR's involvement in these aspects, was formerly limited to displaying a static TIFF. I now use the database features more intensively than I used to - smart collections, publish and so on.

Nothing stays still - not the number OR the nature of the pictures, nor my expectations of my (now aging) computer, nor my expectations of LR, nor the ways I use it and other software.

Like Simon I truly don't see a difference between using relatively smaller and larger catalogs... though I personally am quite selective both with the shutter button and with subsequent triage; and as it happens I don't recall using "continuous" drive mode on either current camera, literally, ever. So a fair sized library in my terms, many others would call still very small.

Z_fan said:

How often you should create new catalogs depends on how much you shoot. A somewhat popular method is to create a new catalog every year. I highly recommend this method if you shoot 50,000 or more images in a single year.

Click to expand...

When a performance ceiling happens, you do something about it - but there are more options, than simply evading the issue by starting afresh arbitrarily... IMO.

If someone's shooting and processing at that rate (more than one shot every six minutes, 12 hours a day, 365 days a year), I would expect him or her to have some more overriding considerations in play, than just what year a picture was taken in. Speculating here, but I'd expect it to be most productive to mirror those considerations, in how you use catalogs.

A folder structure that separates years is an excellent idea, and one can later move whole years physically to other drives etc, with advantage. I'd be looking to divide catalogs though, along personal / business lines, event type, commissioned / uncommisioned, or whatever - once the numbers start to justify this.

I'm not speaking from any personal knowledge of that particular situation, btw - only from managing very similar project and category organisation issues in another field.

It's surely better AFAICT in this situation - many hundreds of thousands of images - to be able to rapidly call up and then filter all of the images you have shot for a given (say) client, spanning all years; than to sort through mixed slices across all the different kinds of work you do, that have each been arbitrarily snipped off at Jan 1st and at Dec 31.

RP