 | This web page has been Archived. Its content will not be updated. Use this link to obtain further details of our archive policy . |
This article presents the general idea of workflow in digital photography, and then focuses on how to deal with different file formats of interest (JPEG, TIFF and raw) using free and open source software. While this subject can be approached from many different starting points, the examples that follow are based on the use of the Ubuntu GNU/Linux distribution, using the GNOME desktop environment. Of course many of the programs discussed are also available for other operating systems such as Windows and OSX.
In my experience, many people assume that The GNU Image Manipulation Program (GIMP), is all a photographer needs if he or she wants to use open source software. Other photographers may assume the photos will look worse or that "my camera would not work in Linux", but both assumptions are misplaced.
My goal in this article is to help computer proficient people understand the needs of a photographer, and to help photographers - from casual snappers to keen amateurs - discover that they can work with free and open source software as well as with proprietary software.
Digital workflow1 is everything you do after you take a photo with your camera and before it is ready to be printed or published.
Fig. 1 shows a typical digital workflow. While image editing is part of the workflow, it can be readily seen that the needs of a photographer cover a wider scope.
I will focus solely on the first two steps of the workflow in this article, i.e. moving files from the camera to the computer and opening them (yellow ellipsoidal area in Fig. 1). While the topic sounds straightforward enough, it raises interesting issues with file formats, in particular with raw files.
Digital cameras store photos as digital files on memory cards in the camera. Most digital cameras can only use one type of memory card, which tend to be one of the following types: Memory Stick™ (MS), CompactFlash® (CF), Secure Digital™ (SD or SDHC), or SmartMedia™. (SM or xD)
Most digital camera makers supply proprietary software applications on a CD along with the camera you purchase. Once installed, this application facilitates automatic detection of the camera when it is connected to the computer, transfer of files from the camera to the computer, conversion from raw to JPEG file formats, setting of parameters such as white balance or exposure, printing and more. Not surprisingly perhaps, many photographers assume that this software is necessary in order to connect their camera to their computer or to perform these functions. In fact this is rarely the case.
As manufacturer programs are seldom available for GNU/Linux, photographers may get the impression that "cameras don't work in linux". But actually, just transferring files depends on the operating system rather than on a particular application. According to H. Figuière, Digital Camera Support for UNIX, Linux and BSD , apart from some very cheap ones, most cameras are supported in Linux. The reason for this lies in standards.
First, most cameras use standard connections:
Second, data transfer is performed with two standard communication protocols, that also make the type of memory card transparent to the computer:
Although archiving and image editing can be performed offline with other programs, some photographers prefer to use a single application that detects the camera, allows basic image editing, file format conversion, archiving, etc., just as the Canon software does. For them, these features are provided by the libgphoto2 library which is available for Linux and MacOS, but not for Windows. Even cameras that implement non-standard protocols are often supported by libgphoto2. The gphoto website has a comprehensive list of digital cameras supported by libgphoto2.
Of course, most photographers are probably more interested in the user-friendly applications that they can use to get the job done, rather than the technical details that have to do with the operating systems or libgphoto2. However, before turning to software applications, it is worth taking a brief look at the question of hardware connections.
Here the memory card stays inside the camera, and the camera is plugged into the computer with a dedicated cable (USB or FireWire). Different cameras need different cables, because although the end that goes into the computer is standard, the camera end changes from model to model.
For example, a Nikon Coolpix 5600 requires a USB Type A Male to USB Mini 8 Pin Male cable (Ref. UC-E6), but a Nikon D70s requires a USB Type A Male to USB Mini Pin Male (Ref. UC-E4 part 25262). The price of a new cable typically ranges from £8 to £15.
This type of connection can be used to operate the camera from the computer using PTP, as explained above, but using a cable has some drawbacks too. Principally, file transfer is very slow and it drains the camera's battery power quickly. You may also find the plastic or rubber lids for the camera connector port to be inconvenient and easy to break, which could leave your camera with an open connector.
A card reader is a simple device that accepts memory cards and is plugged into the computer via USB. They range in price between £5 and £10 and fit any memory card of a certain type (some are multicard).
The file transfer is much faster, no camera battery power is consumed, and they are sturdy and easy to handle. On the downside, the camera can not be operated from the computer.
Digital photos can be stored as JPEG, TIFF or raw files. Each format has different characteristics, and is produced at a different stage of the image formation flowchart as seen in Fig. 3.
Briefly, the characteristics of each format are as follows3:
Showing the difference in image quality is difficult as that can usually only be noticed when zooming in to the image or printing to large format. Below, in Table 2, is a 115x120 px detail from a set of photos at the above quality levels, resized by 400% to 460x480 px with no interpolation, and saved as JPEG with 85% compression. (The detail obtained from the raw image looks softer compared to JPEG and TIFF because it has not been sharpened).
The "Basic" setting displays noticeable JPEG artifacts, that are also visible in the "Normal" setting, but not in the "Fine", "Hi" or "Raw" settings. However, the TIFF file is actually almost twice as large as the raw one. As all the information in TIFF is also in raw (but not necessarily the other way around), at least in this camera it does not make much sense to use the "Hi" setting.
JPEG has been an ISO and ITU-T standard4 since 1994. The most important contribution to the JPEG standard has been attributed to the open source software implementation of the Independent JPEG Group (IJG). JPEG is therefore widely supported and offers no problems for open source software.
Lack of compatibility between TIFF files is not a critical issue for digital photography. As JPEG files are much smaller, they tend to be used in most cases. And if maximum quality is required, then raw is arguably better. However, raw suffers from the same problem as TIFF, impaired compatibility due to proprietary formats.
As indicated above, raw is not a standard format, but a generic way to refer to the way unprocessed data is stored by the camera, and this is why raw files are often considered digital negatives. That is, for raw data to be displayed on screen, processing (e.g. setting the exposure and white balance) is necessary, as opposed to JPEG or TIFF data, that only require decoding (e.g. uncompressing).
So is it possible to work in raw with open source software? In a nutshell: Yes, in most cases, and results might even be better than with proprietary software. But there are limitations and legal issues. These are collectively known as the raw problem.
As discussed above, it is possible to attach the memory card to a card reader, or plug the camera into
the computer with a dedicated cable. The hotplug service will detect the connection and
mount the memory card as a new directory in the file system, e.g. /media/usbdisk-1/. The
digital photo files are accessible by a number of open source software tools that can then be used to
handle the photo files.
For many open source software users, the shell remains the favoured tool to interact with the computer. My experience is that proprietary software users usually hate it with a passion, but for me, the simplest and fastest way to copy files from the memory card to the computer is e.g.
$ cp /media/usbdisk-1/DCIM/100NIKON/* ~/photos/
After the memory card is mounted, GNOME will open a Nautilus file manager window showing the contents of the new directory, as in Fig. 10. Files can then be copied by drag-and-drop or Copy & Paste in the usual way
To see thumbnails of raw files, install the package gnome-raw-thumbnailer
$ sudo apt-get install gnome-raw-thumbnailer
and in Nautilus go to Edit -> Preferences -> Preview, and in "Other Previewable files" change "Only for files smaller than" to 10MB or an appropriate value for your typical raw file size.
dcraw (by D. Coffin) is a stand-alone command line program to read raw files and save them as PPM or TIFF. PPM is a very basic format for graphic files. It is not used by cameras, but merely as a standard format that can be opened by any program. dcraw has become a standard for open source developers. Its code is the base for other tools also reviewed in this section, e.g. F-Spot, Gimp's plugins rawphoto and UFRaw.
$ sudo apt-get install dcraw
The program can then be run from the shell. For example,
produces the file photo.ppm with automatic colour balance.
Let's look at an example. The JPEG files in Fig. 11-12 were created from the raw file produced by a Canon 400d, a fairly recent camera. The first one was generated using Canon ZoomBrowser EX, while the second one was generated using dcraw with default camera settings. The results look quite similar; in my opinion, Canon has better blacks density, while dcraw has better whites and white balance, but it would be very easy to adjust any of the results to match the other.
To convert the file to TIFF instead of PPM do
$ dcraw -a -T photo1.nef
To display the metadata in the file
$ dcraw -i -v photo1.nef Filename: photo1.nef Timestamp: Thu Nov 16 23:13:17 2006 Camera: NIKON E5400 ISO speed: 0 Shutter: 1/6.8 sec Aperture: f/4.6 Focal Length: 24 mm Secondary pixels: no Embedded ICC profile: no Decodable with dcraw: yes Thumb size: 1600 x 1200 Full size: 2608 x 1950 Image size: 2608 x 1950 Output size: 2608 x 1950 Raw colors: 3 Filter pattern: BGGRBGGRBGGRBGGR Daylight multipliers: 2.079447 0.934016 1.229720 Camera multipliers: 1.242188 1.000000 2.417969 0.000000
An important limitation of dcraw is that it cannot decode all the metadata in the raw file. D. Coffin explained this in an interview: ‘ Yes, the metadata is much more complicated [to reverse engineer than the actual sensor data]. That's why dcraw reads only metadata necessary to decode the image, and ignores the rest. ’
rawphoto is the Gimp plugin form of dcraw. The graphic interface is very simple to use. When you try to open a raw file in Gimp, a menu pops up with options to change some default parameters, as in Fig. 13. There are no white balance or curve adjustment options in this plugin.
UFRaw can be used both as a stand-alone application or a Gimp plugin, with the same graphic interface. The graphic interface pops up when a raw file is opened as well, but it offers more options than rawphoto. It is possible for example to adjust white balance, tone curves or exposure.
To process a set of files with user interaction:
$ ufraw *.nef *.cr2
This will open each raw file one by one, and show a graphical interface to allow previewing and tweaking of parameters. Files can be saved as PPM, TIFF or JPEG (preserving metadata).
F-Spot is in fact a personal photo management application. It features file import, image
editing, organizing, and more. It is possible to import photos directly from the camera (F-Spot uses
the libgphoto2 library) or from any directory in the system, including memory cards in a card reader,
external hard drives, a network filesystem, etc. Photos can be saved to directory
~/Photos/ or kept in the original directory. In the former case, a directory tree by
year, month and day is created, e.g.
$ tree Photos/
Photos/
`-- 2006
`-- 11
|-- 16
| `-- DSCN4837.nef
`-- 29
|-- DSCN4827.jpg
|-- DSCN4828.jpg
|-- DSCN4829.jpg
|-- DSCN4830.jpg
|-- DSCN4831.jpg
|-- DSCN4832.jpg
|-- DSCN4833.jpg
|-- DSCN4834.jpg
|-- DSCN4835.jpg
`-- DSCN4836.jpg
Once imported, photos can be managed from the F-Spot main window. Files can be tagged, copied, moved, edited (all versions of an edited photo are saved), removed from the catalogue, deleted or searched by tag or date range.
F-Spot also allows the export of photos to an online sharing service like Flickr, Picasa Web or SmugMug, websites powered by photo album organizers like Gallery and O.r.i.g.i.n.a.l., to a CD or a directory.
Batch processing here means a series of transformations that can be applied repeatedly to a set of images without interaction between the computer and the photographer. For raw files, this usually refers to converting a set of raw files to JPEG using fixed settings. As this is a topic of particular interest to photographers, I have created a new section for it, even though the programs to be used are the same as the ones presented above.
dcraw is intrinsically a batch program, as most command line programs in Linux. For example, doing
$ dcraw -a photo1.nef photo2.cr2
converts 2 raw files from different manufacturers (photo1.nef, photo2.cr2) to PPM. Doing
$ dcraw -a *
UFRaw provides the program ufraw-batch (in Ubuntu both are bundled in the same package) for batch processing of raw files from the shell. The parameters for the image conversion can be set in the shell as usual
$ ufraw-batch --exposure=3.0 *.nef
But it is also possible to use the graphic interface of ufraw to set those parameters, and save them in a so-called ID file. For example, Fig. 17 shows the interface after opening photo1.nef, changing the exposure level to 3.0 and selecting the option to save an ID file.
This generates a text file called photo1.ufraw that contains the conversion parameters:
...
<Green>1.245920</Green>
<ChannelMultipliers>1.000000 1.126845 3.243823</ChannelMultipliers>
<Exposure>0.300000</Exposure>
<OutputType>4</OutputType>
...
It is then possible to process a batch of files using these parameters from the shell
$ ufraw-batch --conf=photo1.ufraw *.nef ufraw-batch: loaded photo1.nef ufraw-batch: saved photo1.jpg ufraw-batch: loaded photo2.nef ufraw-batch: saved photo2.jpg
This article has presented the idea of workflow in digital photography and then focused on technical and practical aspects of dealing with JPEG, TIFF, and raw files using open source software.
In practice, it is feasible and simple to download photo files from the vast majority of cameras onto the computer, either using a dedicated cable or a card reader for the memory card. Raw files are well supported; this support is even integrated into the file manager of the GNOME desktop. There are also programs that can convert raw files to JPEG or TIFF, either in interactive or batch mode.
It is licensed under the
Creative Commons Attribution-ShareAlike 2.0 England & Wales licence.
OSS Watch is funded by the Joint Information Systems Committee.