Automated JPEG copy and compress

[The late XV105]

The computing knowledge and skills in this forum never cease to impress me, so here goes with another challenge.

Background
As some of you will recall, for media streaming I have a partioned 1TB USB HDD connected to a NAS on a gigabit home network. Included in this media is a copy of my entire digital photo collection, built, file structure included, by periodically running Karen's Replicator against the root of the photo directory structure on my media server.

This is great because it means the photos are always available on line and the original source is never put at risk. If something gets deleted or moved from the copy, it gets corrected the next time I run the replicator (which is net change, so fast, too). I have a multiple layer backup strategy as well but that's outside scope here.

From this you will correctly understand that the copy is exactly that. A copy. I can do what I like with it.

The described copy of my photo collection is now usually used as source for e-mailing or for wirelessly browsing pictures on one of the laptops or the netbook in the household. This means that a typical 5-8MB JPEG file is inappropriate in the former case and is a waste in the latter as the screens being used don't have high enough resolution to take advantage of the available detail. It's also a PITA in the latter case because each file takes many seconds to load at a maximum of 54Mbps (versus the blink of an eye on any of the gigabit LAN enabled devices!)

For e-mailing we have historically used the XP PowerToy for picture resizing with great success; even selecting "large" (1024x768 resolution) will typically reduce a 5MB JPEG to less than 100Kb. Perfect for e-mailing, and it occurred to me last night, for snappy browsing wirelessly.

Question
What I want to do is obtain or develop a script that automatically performs the net change file copy currently performed by Karen's Replicator AND which runs a Jpeg compression equivalent to the XP PowerToy.

How, please?


Cheers!
XV

[bnt]

For scripted image operations, the major player is ImageMagick, which is free. Have look at their command line processing page for an idea of what it can do. It can convert multiple files at once. You could add the relevant commands to the script before running the replicator.

If you're looking for a replicator, and it's an all-Windows setup, then Microsoft's SyncToy is fine, and now includes a good command line / scripting version.

[The late XV105]

Thanks very much, bnt.
I'll take a look at ImageMagick right away.

If Microsoft's SyncToy can be invoked* as part of the same script that invokes ImageMagick I'll take a look at that too, else I'll stick with Karen's Replicator given that it works flawlessly and has all the configuration options bar scripted invoking that I need.

Cheers,
XV


*Ideally from one script I want to scan for new photos and for each that is found, as it is found, resize and save the result on the mentioned network drive whilst leaving the original untouched.

[mixture]

Microsoft's SyncToy

Heh, I wouldn't trust any Microsoft software where they've explicitly decided to call a toy !

One word.

Robocopy.

Ideally from one script I want to scan

You can do whatever you want all in one go from a batch file assuming the target software has a CLI API...

[The late XV105]

Heh, I wouldn't trust any Microsoft software where they've explicitly decided to call a toy !

Seriously though, the Jpeg resizer I currently use is also a Microsoft (Power)Toy, released as an unsupported tool that was ostensibly written as a personal project and it's been utterly reliable!

Thanks for the Robocopy tip - I'll take a look, and thanks too for the CLI script comment; it's what I would assume but I haven't yet confirmed how ImageMagick is invoked.


Cheers,
XV

[bnt]

Robocopy is great if you have a "master / slave" setup - one side must always mirror the other side exactly. SyncToy can do bi-directional sync (no master), which is what I thought of when I read "replicator".

You want to process "new photos" ... if I was doing something like this, I might do it in a "batch" fashion, moving files around according to their status. New photos land in one directory and get processed e.g. hourly. Use ImageMagick to process all the new files, with the copies landing in the network drive. I'd move the new originals to an archive after processing, perhaps by name. That way, you can just run ImageMagick on all the files in the directory, rather than trying to detect the new files in a crowded directory.

[mixture]

SyncToy can do bi-directional sync

I hope you can spell corruption bnt.

Consumer grade synchronous replication tends to be a recipe for disaster in my opinion. Quorum and tiebreak implementation tends to leave a lot to be desired !

As I said, just my own little opinion (in general, not just picking on M$), your mileage may vary, of course !

[mixture]

XV,

Rsync for Windows might also be worth a gander ..... might also do bnt's bi-directional replication (it's been a little while since I used it, and never used the Win version)

cwrsync - Rsync for Windows | ITeF!x

Rsync uses the Rsync algorithm (duh !) .... which only sends deltas accross the link.... leading to fast efficient transfers.

[The late XV105]

I already have a process, thanks, bnt

My primary process:

1. Download RAW images from card to a dedicated "Received" folder on a dedicated "WIP" HDD. This disk is NOT backed up to avoid file locking and performance problems especially with the large video files also edited
2. Perform all RAW editing
3. Produce JPG copies of all RAW files that I haven't deleted
4. Perform all JPG tweaks
5. Perform a self-critical slide show of the result
6. Re-do any editing if required
7. Re-perform a self-critical slide show of the result, if required
8. Copy all JPGs to their respective final resting place in a logical file structure hierarchy which is on another HDD
9. Observe from System Tray icon that Memeo AutoBackup has woken up
10. Monitor Memeo AutoBackup and wait until all three backups* have finished automatically running
11. Manually verify the NAS and USB backups by random sample
12. Delete all RAW files from WIP/Received
13. Delete al JPG files from WIP/Received
14. Reformat memory card

Secondary process, once a week or so

1. Start Karen's Replicator
2. Run a saved photo net change job that copies only photos added since the last run to the USB HDD connected to the NAS
3. Verify when done that the root folder shows the same file count and total size as the source root folder

It is the latter process that I simply want to modify to include JPG compression


XV


*eSATA HDD which serves as a convenient backup (no fire or theft protection) if I have a primary HDD crash, RADI1 NAS in the garage via gigabit network, and external USB HDD swapped periodically with another kept off site.