Building an advanced photographic archive

Cardbox comes with a sample database for storing and indexing photographs. You can scan in full-sized photographs or simply thumbnails; and it's an easy matter to add fields for indexing and classification. You can also design big formats for viewing the photographs or small ones for summary listings. The small formats can omit the photographs or present them as miniature thumbnails.

All this, you can do with Cardbox on its own. Nothing else is required.

The bigger picture

Most of the time this is all you will need, and Cardbox will do everything you want. But if you have a huge archive with thousands of photographs, and you want to throw away the originals and store your photographs digitally at high resolution, then your database will be enormous. Cardbox won't mind, but you may worry about having enough space to store it or about how you are going to keep it backed up.

In that case, the sample we present here may be what you are looking for. Instead of storing the full-sized photographs in your Cardbox database, it stores them in the Cloud, on Amazon's S3 Internet storage system, while the Cardbox database just has a screen-sized version of each picture. So you can search and view your photographs easily using Cardbox, and when you want to print a picture or manipulate it in Photoshop, you can download the full-sized version from S3.

Software required

You need Cardbox.

If you don't already have Cardbox, download and install it now.

Downloading and installing the sample database

The sample database, together with all the necessary macros, is available for download in a file called photoarchive.zip.

Choose the folder where you want to store this database: use Windows Explorer to create it if necessary.
Click on this link and tell your browser to download and save this file.
Once your browser has downloaded the Zip file it may ask you whether to run (or open) the file: tell it to do so. If it doesn't ask, you'll have to locate photoarchive.zip and double-click on it yourself.
Within the photoarchive.zip folder, highlight everything (using the mouse, or by pressing Ctrl+A) and then press Ctrl+C to copy all the files to the Clipboard. Don't close the folder yet.
In a separate window, navigate to the destination folder you chose in step 1, and open it.
Press Ctrl+V to copy all the files from photoarchive.zip to your destination folder.
You can now close the photoarchive.zip folder.

Signing up for Amazon S3

You need to open an account with Amazon Web Services before you can store data in Amazon S3. Visit the Amazon S3 page and follow the instructions.

When you have signed up successfully you will receive two pieces of information from Amazon: the "key ID", which identifies your account, and the "secret key", which lets Cardbox convince the Amazon server that it is working legitimately on your behalf. Keep both these pieces of information safe (we store them in our encrypted Passwords database).

Setting up Cardbox to use your Amazon S3 account

Open Cardbox and use the command Tools > Management > Amazon S3.

Leave the host name unchanged and fill in your key ID and secret key. Tell Cardbox to remember them. Then click on the Next button.

Creating a bucket for storage

You will now be on the Select/Manage Buckets page. A "bucket" is a subdivision of Amazon's S3 storage and every piece of data has to be stored in a bucket somewhere. Think of a name that seems reasonably mnemonic to you, type it into the "Bucket:" box, and click on the Create button.

Bucket names have to be unique across the whole of Amazon S3. If you choose a name that has already been taken (such as "photographs") then an error message will pop up and you'll have to choose a different name.

Once you've created your bucket, click on the Cancel button because you don't need to bother with the next page.

Telling Cardbox which bucket to use for the archive

Within Cardbox, open the PhotoArchive database.

Use the command Special > Set Bucket Name. The command will ask you for a bucket name: enter the name in exactly the format that you used in Select/Manage Buckets. This includes using the same case: "Bucket" and "bucket" are two different names. If you enter the wrong name then the command will report an error

Adding photographs to the database

(We assume that you still have the PhotoArchive database open.)

Use the command Special > Upload and Import Photographs.

Cardbox will pop up a standard Windows "File Open" command. Navigate to the folder where your images currently are, and highlight the image files you want to process. If you want to highlight every file in your chosen folder, pressing Ctrl+A should do this for you. Then hit the Enter (Return) key or click on the Open button.

Cardbox will read each file in turn, upload it to your Amazon S3 storage and also store a miniature copy of it in your Cardbox database. The progress bar at the bottom of the screen will show you how far the process has gone.

Speed

Unless you have a really fast Internet connection the upload process can be quite slow. This is especially the case with ADSL. The A in ADSL stands for asymmetric and it means that the speed of uploads to the Internet is a small fraction of the speed of downloads from the Internet. In our experiments it looked as if a two-megabit-per-second Internet connection gave you an upload speed of just over two megabytes per minute - and with some image files being almost 2MB in size, that can take a long time!

Don't be too unhappy about this:

You only need to do this once.
You can do other things while it's going on.
You can leave your computer running overnight.
If you want to interrupt an upload, click on the red X next to the progress bar. Next time, when you want to continue uploading, make sure that you select only the image files that have not already been uploaded. This is easy: just press Ctrl+End to go to the last record in the PhotoArchive database and you'll see the last image file that was uploaded.

You might also like to know that once you've uploaded the files your troubles are over: downloads using ADSL are about 7 times faster than uploads.

(For comparison, an upload from a directly-connected server in Rackspace's data centre in the UK took around 40 minutes per gigabyte - a figure that will be made over 10 times faster once Amazon start to deploy S3 servers in Europe).

After you have added the photographs

To view a high-resolution photograph in your web browser, press the View button in the photograph's Cardbox record, or move onto the record and then use Special > View Current Image.

To download a high-resolution photograph directly without going through the web browser, press the Download button or move onto the record and then use Special > Download Current Image.

Allowing someone else to download

To create a link (URL) that allows someone else to download one of your high-resolution photographs (without giving them your Amazon keys), move onto that photograph and use Special > Create URL. The link will be copied to the Clipboard, from where you can paste it into an email. You can choose how many days the link will be valid, or you can create a link that lasts for ever.

Indexing

This sample is intentionally very bare. You will probably want to annotate your records with all sorts of information - genre, circumstances, project, client, subject, and so on. The great thing about Cardbox is that you can add fields to your record designs at any time. Whatever information you want to store and search on, you can create a field for it - and this is an ideal opportunity for you to get some practice! You might like to look at the sample Photos database on our Cardbox Server: do File > Open > Server, select or type database.cardbox.net on the left-hand side of the window, press View, and double-click on the Photos database to open it.

Also remember the Batch Edit command, because it is very useful if you want to add the same piece of information to multiple records. If you want to label a batch of records with (let us say) a subject name, it is much easier to:

Look through records and hit Tab to tag the ones you want to label.
Use Tag > Select to select the tagged records.
Use Edit > Batch > Edit and edit the first record in your selection.
Let Cardbox edit the remaining records automatically.

than it is to edit one record after another by hand.

Assumptions we have made

We've made a number of assumptions, any of which you can change simply by editing the macros we have provided.

You want to have miniature versions of the photographs on your computer: If you don't, just edit the macro to remove the ReadFromFile command. It will make the database smaller, too.
You want your miniature version to be 480 pixels across: If you want it larger (for instance, 800 pixels) or smaller (for instance, 64 pixels), edit the macro.
You want the folder name within your bucket to be "archive": You can change the name to be anything you want.
Your image files are readable as images by Cardbox: If they aren't, remove the ReadFromFile command and do without the miniature photographs.
Your photographs have unique filenames: If they don't - that is, if two different photographs in different folders can have the same filename - then you will have to change the macro to make sure that each uploaded photograph file is given a unique name.
All your photographs are in one folder: In fact, they quite often aren't; but usually it's acceptable to process one folder at a time, given the time that the upload takes. If you want to process more than this then you have two choices: rewrite the macro to scan more folders, or temporarily copy all your photographs to a single folder and delete that folder after you've done the upload.

Things we haven't done

This is an example to show the capabilities of Cardbox. There are many things that we could have done but haven't. For example:

Summary formats - You can create alternative formats for viewing your records differently - for instance, a one-line summary format without images that can display multiple records to a page.

Batch download - You can write a macro that downloads many files at once, rather than requiring you to click the Download button for each file separately.

Deletion - Deleting a record will not automatically delete the uploaded file. It's easy to write a macro that does.

Batch deletion - An alternative would be to delete all uploaded files that aren't mentioned in the database. Again, a macro can be written to do this.

Record rescue - The opposite can be done equally easily: a macro that looks for uploaded files that aren't mentioned in the database, and adds database records for them.