The Lightroom catalog
Martin Evening | Software Techniques | Published Dec 28, 2012
|Adobe Photoshop Lightroom is built atop a database architecture that relies on a centralized catalog to house information about your still images and video files.|
One of the most common questions I'm asked about Adobe Photoshop Lightroom from new (and sometimes not so new) users is, 'Why do you have to import photos to the catalog first before you are able to edit them?' The answer to this question goes right to the heart of Lightroom's approach to information storage and retrieval, which to my mind more adequately addresses the needs of today's photographer. In this article I'll lay out the basic principles of image management in Lightroom and explain how this approach can actually make the task of locating your images more efficient.
Image browser vs. database manager
As computer users who've come of age during the PC revolution, we have grown accustomed to the idea that everything needs to be sorted into folders, and indeed folder hierarchies have become the primary means of organization. This file directory management approach may make sense with task-specific Word documents but becomes extremely limiting when applied to large collections of less easily-defined images. Yet this is exactly how image browser software, like Adobe Bridge operate. For all their admittedly useful features, they simply browse the existing folder structure on your hard drive.
And my chief complaint about sorting images by folders is the very real problem of determining just which folder they should go in. Someone once told me about an underwater photographer who maintained a small photo library of his work. When adding new transparencies, he would have dupes made so that a photograph of say, a diver with a shark could be filed in one set of physical folders labeled ‘sharks’, another named ‘divers’ and another broken down by ‘location’. This physical duplication was necessary in order to make the library system work effectively, but no one would call it efficient. Yet there are people who work this way with digital files precisely because they are not using a database-driven management system.
|Here is an example of an underwater photo (in this instance a photo shot by Jeff Schewe). It makes sense to categorize this by the location it was shot in, the presence of a diver, as well as the coral featured in the foreground. In the folder-based example I mentioned above this could involve duplicating the master image several times. Using digital asset management software such as Phase One Media Pro or Lightroom, however, there is no need to create physical duplicates.|
I can't count the number of times I have sat through a seminar where the instructor has come unstuck when relying on folder/browser navigation to locate their demo files. Meanwhile, the audience waits impatiently while the instructor sifts through a complex hierarchy structure of folders known only to himself. Sound familiar?
Database management programs designed specifically for photographs were developed to tame such chaos. One of the first and most well-known was iView Media Pro. The company behind it was acquired, first by Microsoft which renamed it Expression Media, but is now developed by Phase One and sold as Phase One Media Pro. Extensis Portfolio was another option available, but it was the launch in 2005 of Aperture from Apple that showed how one could combine the power of a database with image editing tools to provide an all-in-one solution for photographers. Adobe's entry, in the form of Lightroom, launched soon after.
|Point a browser-based app like Adobe Bridge (shown here) at a folder on your hard drive and you'll see everything that’s on the computer.||By contrast, the Lightroom Import dialog, pointed at the same folder, only shows the three JPEGs files located there.|
What all these database-driven apps have in common is that you have to explicitly import media files into the program, adding them to an app-specific catalog, which is the primary holder of all your image data. This process, while it may seem an unnecessary step, is required in order to build a catalog of files made up exclusively of the images and videos you have deemed relevant. One under-appreciated aspect of the import process is that when you select a directory containing many different types of data, i.e., music files, spreadsheets or Word documents, the software filters out any file formats it does not support. This can work to your advantage as you only see the files you'd actually consider working with and don't have to bother wading through irrelevant files.
Metadata: Your new best friend
Instead of relying on a folder structure to sort your images you can manage your photos much more effectively by using keyword and EXIF metadata. Metadata - literally data about your data - is vital to managing any collection of digital files. In fact, placing an image inside a descriptively named folder amounts to a very rudimentary type of metadata; an identifier that can be used to help locate the data stored in that folder.
Keywording is a very robust form of user-generated metadata that allows you to apply multiple descriptive tags to an individual file. Let’s return to the underwater photographer example I mentioned earlier. This guy could choose to replicate his film library system by placing duplicate image files in multiple folders that could be read by a file browser. With a database-driven cataloging program, however, he would only need to import one master image and then use the cataloging software to append multiple keyword metadata tags to identify the image. With a program like Lightroom it is fairly easy to create varied collections of photos in which multiple instances of a single master photo appear in more than one collection. No files are duplicated on your hard drive. The single master image is simply referenced multiple times by the catalog. Edit that master image later on, say by converting it from color to BW, and the new version is automatically propagated in every collection. That's about as close to a free lunch as you'll get.
|Here is a photo of a male model I shot in Italy. My Lightroom catalog contains only a single master file but the image belongs to five separate collections (virtual folders). You can't do this in a folder-based management system without creating duplicates, or at least aliases, of the original digital file.|
Crucially, once you start getting into the habit of tagging your photos with metadata, it then becomes easier to find them. Imagine trying to locate an image on someone else’s computer using a folder browsing method only. You could start by looking inside the My Pictures folder, but where would you look next? For that matter how successful are you at navigating your own Pictures folder? We have all struggled at times to locate a specific file that we know is on the computer system somewhere.
With keywording the task becomes simple. If you know how to do a Google search you already know how to search by metadata. In my opinion, this is where the database/catalog method of managing your image library makes your life much easier.
If the very idea of manually applying keywords is making you dread going through an image library consisting of hundreds or thousands of images, don't worry. Lightroom gives you the ability to save commonly used keywords as one-click templates that can be batch-applied to as many images simultaneously as you desire. But it gets even better. Every digital camera file comes with some metadata tags of their own that have been embedded by the camera that created it. Called EXIF data, this includes descriptions such as exposure settings, camera name, capture date/time, lens and focal length, image dimensions, even GPS coordinates if your camera records it.
Image file vs. Catalog file
By default, Lightroom stores its metadata, including any adjustment edits you make, not inside the image files themselves, but in an actual catalog file residing on your hard drive. This is a database file that summarizes everything referenced in the catalog and is continually updated as you edit those images. This database file is designed to provide fast access when carrying out a metadata search. You can choose to manually select a file or group of files in Lightroom and save this metadata to the XMP header space of a JPEG/TIFF/PSD file or a .xmp sidecar file for proprietary raw file formats (Command/Control+S is the shortcut). You can even configure the Lightroom preferences so that the files are updated automatically in the background.
The main thing to appreciate about the catalog file is that it is a single, compact document which summarizes everything Lightroom needs to know about the its library contents. How is this centralized approach helpful? With an image browser such as Adobe Bridge, any metadata change you make, even something as simple as adding a single keyword, updates your entire image file. That means that when you do your (hopefully daily) backups, you must backup every single image file you've modified with a metadata change.
Performing the exact same modification in Lightroom, however, all you need to do is copy a single catalog file to keep your backup up-to-date. With just one relatively small document to worry about, it is even feasible to keep a backup copy of the catalog file in cloud storage, such as is included with an Adobe Creative Cloud subscription.
|Lightroom's data files are stored by default in the Pictures folder (Mac) or My Pictures folder (Windows). The most important document is the .lrcat catalog file (on the left). This file contains all the essential catalog information and needs to be carefully safeguarded. Without it, the only metadata information you will be able to access will be that stored in the files themselves.
The previews.lrdata file is also important as this will contain an archive of rendered previews for the catalog files. Lightroom uses this so it doesn't have to rebuild previously rendered image previews every time you launch the app. So you don't want to lose this either, as it can take a long while to regenerate a large one from scratch, but it can be done, as long as the catalog file is salvaged.
The Backups folder is where Lightroom-generated auto catalog backups (named by date) are stored.
So even though Lightroom gives you the option of automatically writing metadata changes into XMP (see the screenshot below), I believe it is best to keep this data stored centrally in one place: the catalog file. No matter how rigorous you are about saving the metadata, the catalog file will always contain the most up-to-date information about the metadata in all the catalog files. One exception I'd make would be if you need to maintain compatibility between Lightroom and Bridge. In such case, it is essential you have Lightroom write the changes into XMP in order for Bridge to be able to read the Lightroom edits.
|If you open the Catalog Settings (Lightroom > Catalog Settings…) this opens the dialog shown here where there is the option to enable 'Automatically write changes into XMP'.|
If you've read this far, it should be obvious that maintaining regular, current backups of the Lightroom catalog file is absolutely essential. In the event that your Lightroom library stops working properly the catalog file is always the first thing you’ll want to salvage. This can be done by either running a repair on the current catalog or reverting to the most recent backup version. This can be a backup catalog you created manually when prompted to do so by Lightroom, or it might be one that was saved during a regular, scheduled system backup. Keep in mid that even if you've been automatically saving metadata changes into the XMP of individual images, certain catalog information such as collections and virtual copies are stored exclusively in the catalog file.
In summary, the Lightroom catalog approach is more economical compared to browser viewing and offers a more robust approach to the management of image data. While it is possible to do most of what can be done in Lightroom using an image browser like Adobe Bridge, hopefully the insights I have provided here show why the catalog approach is more versatile and can ultimately offer you greater potential security. Whether you're a longtime photographer or just starting out, we can all expect our photo libraries to mushroom in size in the coming years. Therefore, now is a good time to be thinking about the most suitable strategy for managing your photos – not just for today, but long into the future.
Martin Evening is an award winning advertising and fashion photographer based in London, England. He is also a best-selling author of instructional titles such as The Adobe Photoshop Lightroom 4 Book and Adobe Photoshop CS6 for Photographers.