Proposal for "quasi deletion" of image files on an SD card

boblite said:

Several commenters have said they find my workflow illogical... have asked why I don't just copy all pix from the SD card to the PC and sort them out there.

To recapitulate, my desired work flow is as follows:

[1] Delete bad pix on the SD card using a fast editor like Irfanview

[2] Copy remaining pix for later categorization and possible editing with software that has slow or limited photo management but good photo processing.

My reason for doing things this way is that, as I note elsewhere, I take huge numbers of pix, about 90 per cent of them bad.

I don't want to flood my current choice of photo editor (Corel Paint Shop Pro) with giant numbers of pix because in my experience it does not handle giant numbers well. Also, I want to make it easy to use other photo editors because I've already purged bad pix and have a much smaller subset to deal with.

P.S. I don't reformat SD cards. I keep them as backup. Used cards would even fit in my bank safe deposit box which has severe space limitations.

P.P.S. Rather than delete files from the SD card it would be nice if there were a dedicated ultra fast photo manager where as each picture pops up, the user can type a code like 1 for excellent, 2 for acceptable, 3 for bad, and the first character of the file name is replaced with the code and the next pic pops up very fast. This would provide a sort of preliminary triage. Then one would copy all the files from the SD card to the PC. And process the desired subset. WIthout throwing anything away. There might come a day when a bad shot might contain valuable information........

Click to expand...

Maybe Photo Mechanic is the software you are looking for?

https://en.wikipedia.org/wiki/Photo_Mechanic


For me, I only shoot my daughters at their indoor sports along with their teams.

Something like this seems like overkill for me.

And . . . my process of "write-protecting" my keepers in-camera works for me and . . . is a free way to accomplish my goal.

Take care & Happy Shooting!

From what you've said I get the impression you are a professional, with deadlines, where you really need to get the stuff out the door ASAP. In my own case I am just an amateur with no deadlines, with lots of other stuff needing attention.

I tried a bit of in-camera deleting when sitting in an airport cellphone parking lot waiting for a friend to arrive. What I did not like is that the LCD monitor is too small to reliably tell me whether a shot is worth keeping, except in cases where the shot is egregiously bad.

Maybe it's possible to do in camera editing with an Ipad, for wi-fi equipped cameras, but I just don't often find myself in a situation where this would make sense. I do all my editing comfortably seated in front of my fast workstation PC with a giant monitor where I can see each pic in great detail.

On the other hand.... some cameras claim all kinds of powerful in camera editing.... especially of RAW files ... which I have not tried... have you been able to do this ? And would it be possible to do this with a Windows 10 PC ?

Cheers

Bob

Just saw the above post where you say you're not a pro... that you're taking pics of your daughters' sporting events....

Photo Mechanic is $150 and more than I would care to spend. But thanks for the tip.

Cheers

Bob

How fast do they become unreliable ? Presumably this varies by card... As far as archival medium goes, 10 year survival would be just fine for my purposes.

boblite said:

From what you've said I get the impression you are a professional, with deadlines, where you really need to get the stuff out the door ASAP. In my own case I am just an amateur with no deadlines, with lots of other stuff needing attention.

Click to expand...

I do do paid work.

For paid work, I do portraits. I actually take a lot of pictures, but a lot less than when shooting my daughters at cheer competitions.

So for paid portrait shoots, I actually copy all pictures to Lightroom.

But for when I am shooting my daughters' cheer teams, and all the other teams at the club, I end up with thousands more images than a paid portrait session.

That is why I worked out this trick with "write-protecting" the files in-camera.

boblite said:

I tried a bit of in-camera deleting when sitting in an airport cellphone parking lot waiting for a friend to arrive. What I did not like is that the LCD monitor is too small to reliably tell me whether a shot is worth keeping, except in cases where the shot is egregiously bad.

Maybe it's possible to do in camera editing with an Ipad, for wi-fi equipped cameras, but I just don't often find myself in a situation where this would make sense. I do all my editing comfortably seated in front of my fast workstation PC with a giant monitor where I can see each pic in great detail.

On the other hand.... some cameras claim all kinds of powerful in camera editing.... especially of RAW files ... which I have not tried... have you been able to do this ? And would it be possible to do this with a Windows 10 PC ?

Click to expand...

I actually don't edit the pictures in-camera.

I am only picking which images I want to edit in Lightroom when I get home.

So, there is always a delay from when I take the pictures and I actually hand them to my wife to post to the club's Facebook page.

But this trick of selecting images by "write-protecting" them in-camera has reduce my turn around time from 1-2 weeks down to 2-4 days.

The only way I could make this faster is to start editing the images while we are still at the competition.

And . . . I have thought about how to do that. I could get a Windows 10 tablet and install Adobe Lightroom and Photoshop on it. I already pay for that software for my paid work. And I believe you can have it installed on 2 computers.

So . . . I would still select pictures by "write-protecting" them in-camera. And then only move those images over to the Windows 10 tablet to start editing while I sit there between runs.

Take care & Happy Shooting!

boblite said:

Cheers

Bob

Click to expand...

boblite said:

How fast do they become unreliable ? Presumably this varies by card... As far as archival medium goes, 10 year survival would be just fine for my purposes.

Click to expand...

10 years is roughly the lifespan presuming average amount of writes.

https://www.sdcard.org/consumers/faq/index.html#servicelife

A SD card (and all types of memory) wears mainly based on how much writing is done on it, so you may get less than 10 years or more than 10 years depending on how much writing is done.

boblite said:

Just saw the above post where you say you're not a pro... that you're taking pics of your daughters' sporting events....

Click to expand...

+1

I actually do do paid portrait work.

Ironically I take less pictures at a portrait session than at cheer competitions.

But the difference is . . . I slow things down at portrait sessions and my wife and I have things worked out as far as what we are going for.

For cheer competitions, I shoot first and ask questions never. It's a competition that doesn't happen again. It's more productive to take the shot and throw it away (in the digital age) than to kick yourself for missing the opportunity of taking the shot in the first place.

That and I try to get shots of as many of the athletes from the club as possible. So upwards of 100-175 athletes, depending on which teams are attending that weekend's competition.

boblite said:

Photo Mechanic is $150 and more than I would care to spend. But thanks for the tip.

Click to expand...

+1

It seems like overkill for what I want to do with it. And more work as well.

It would entail having a computer there, being able to pull the SD card out of my camera. Tossing it into the computer. Chugging through the images. And then remembering to pull the SD card back out of the computer, to toss back into my camera. Putting the computer away . . . all before the next team runs up for their run through. And I have to spend $150 on Photo Mechanic + $500-1000 on a tablet PC to run it on.

Or I could simply look at the back of my camera and "write-protect" the images I want to keep.

To me, one choice seems to be cheaper and easier to implement for what I want to do. LOL.

But . . . I can understand why pros that submit photos (especially at sports events) like Photo Mechanic. The ability to run through images quickly and add tags quickly can make or break your picture being picked up for publication.

Take care & Happy Shooting!

boblite said:

Cheers

Bob

Click to expand...

--
My Personal Flickr Favs . . .
http://www.flickr.com/photos/tacticdesigns/sets/72157631300869284/
[FL][RP][LS][GC][51][ML]

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed but the data remains until it is overwritten.

Thanks so much for the link.

Getting down into the weeds, it occurs to me that taking a pic counts as a major write taking many megabytes. Deleting a pic is probably just changing a few characters in the file table (??) leaving the file itself unchanged. Under a computer OS the space occupied by the file would be available to subsequently created files. I would suspect that the camera OS would not try to do that and would just leave the space occupied by the deleted file unused.

If one never reformats and just keeps taking pix until there's no space left... and deleting unwanted pics... and then stops using the card.... seems to me that would involve much less wear than frequent reformatting.

boblite said:

Thanks so much for the link.

Getting down into the weeds, it occurs to me that taking a pic counts as a major write taking many megabytes. Deleting a pic is probably just changing a few characters in the file table (??) leaving the file itself unchanged. Under a computer OS the space occupied by the file would be available to subsequently created files. I would suspect that the camera OS would not try to do that and would just leave the space occupied by the deleted file unused.

If one never reformats and just keeps taking pix until there's no space left... and deleting unwanted pics... and then stops using the card.... seems to me that would involve much less wear than frequent reformatting.

Click to expand...

Seems there is a lot of over thinking for what is a very simple task.

boblite said:

Thanks so much for the link.

Getting down into the weeds, it occurs to me that taking a pic counts as a major write taking many megabytes. Deleting a pic is probably just changing a few characters in the file table (??) leaving the file itself unchanged. Under a computer OS the space occupied by the file would be available to subsequently created files. I would suspect that the camera OS would not try to do that and would just leave the space occupied by the deleted file unused.

If one never reformats and just keeps taking pix until there's no space left... and deleting unwanted pics... and then stops using the card.... seems to me that would involve much less wear than frequent reformatting.

Click to expand...

You don't have to worry about it, the SD card itself actually has its own controller that manages where each write occurs in order to spread out the wear. This is independent of the file system.

For taking pictures, the load on the SD card is fairly mild, so you don't have to worry. It'll likely last over 10 years especially if you don't fill up the card every day. Continuous video (like for example security cameras that use SD cards as backup and continuously loop) will be the biggest load.

mdmarqphoto said:

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Very interesting.... nice to know. Thanks.

Thanks for this info. Nice to know.

Joep van Steen said:

mdmarqphoto said:

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Click to expand...

Yes it is, that's why "undelete" programs can recover files. Only the file name is changed in the FAT (to indicate the file has been "deleted"), the rest of the file name and the location of the file data on the media is still present.

mdmarqphoto said:

Joep van Steen said:

mdmarqphoto said:

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Click to expand...

Yes it is, that's why "undelete" programs can recover files. Only the file name is changed in the FAT (to indicate the file has been "deleted"), the rest of the file name and the location of the file data on the media is still present.

Click to expand...

Oh, oh; I hear a rumblin' in the distance...

mdmarqphoto said:

Joep van Steen said:

mdmarqphoto said:

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Click to expand...

Yes it is, that's why "undelete" programs can recover files. Only the file name is changed in the FAT (to indicate the file has been "deleted"), the rest of the file name and the location of the file data on the media is still present.

Click to expand...

You're wrong on a few points.

File name is not kept in the FAT. If we limit ourselves to FAT based file systems, the file name is part of a directory entry. As long as we have the directory entry we have the file size + start cluster (only!)

FAT entries for the deleted file are zeroed.

Indeed actual file data isn't deleted.

The directory entry + clusters previously allocated to the file may be re-used at any time, so you need to recover the files before anything get's written to these clusters and/or entry is re-used.

An undelete program can recover deleted files because indeed initially data is not wiped, and as long as data was stored in a contiguous block of clusters. Once the directory entry is lost data may be recovered by scanning for 'file signatures' (for example FF D8 FF in case of JPEG) at cluster boundaries.




FAT before deletion




FAT after deletion



--
Joep

Joep van Steen said:

mdmarqphoto said:

Joep van Steen said:

mdmarqphoto said:

boblite said:

What I am proposing is "quasi deletion".

For example, suppose the end user who is reviewing photos on an SD card using a computer has access to a button that says "mark photo as unusable", which when pressed causes the computer to change the first character of the file name on the SD card to a ~ (tilde) character.

Click to expand...

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Click to expand...

Yes it is, that's why "undelete" programs can recover files. Only the file name is changed in the FAT (to indicate the file has been "deleted"), the rest of the file name and the location of the file data on the media is still present.

Click to expand...

You're wrong on a few points.

File name is not kept in the FAT. If we limit ourselves to FAT based file systems, the file name is part of a directory entry. As long as we have the directory entry we have the file size + start cluster (only!)

Click to expand...

https://www.forensicswiki.org/wiki/FAT

"Recovering directory entries from FAT filesystems as part of recovering deleted data can be accomplished by looking for entries that begin with a sigma 0xe5. When a file or directory is deleted under a FAT filesystem, the first character of its name is changed to sigma. The remainder of the directory entry information remains intact. "

https://www.minitool.com/data-recovery/fat32-data-recovery.html

"Here is how a file is deleted under Windows OS (operating system):

No matter a file is deleted by users on purpose or unintentionally, the first character of this file in FDT will be changed to "E5", which indicating deletion; the corresponding information saved in the second sector of boot sector and showing the disk space occupied by certain partition, on which the deleted file is stored, will be changed.

As a result, under the condition that new file is not saved to the place where deleted file was located, you can recover deleted files from FAT32 drive correctly by modifying corresponding location and searching for E5 deletion mark according to data storage structure and data link of deleted file ."

mdmarqphoto said:

mdmarqphoto said:

mdmarqphoto said:

mdmarqphoto said:

mdmarqphoto said:

FYI, that's how files are deleted already on Windows based systems, only the first character of the file name is changed

Click to expand...

It isn't.

Click to expand...

Yes it is, that's why "undelete" programs can recover files. Only the file name is changed in the FAT (to indicate the file has been "deleted"), the rest of the file name and the location of the file data on the media is still present.

Click to expand...

You're wrong on a few points.

File name is not kept in the FAT. If we limit ourselves to FAT based file systems, the file name is part of a directory entry. As long as we have the directory entry we have the file size + start cluster (only!)

Click to expand...

https://www.forensicswiki.org/wiki/FAT

"Recovering directory entries from FAT filesystems as part of recovering deleted data can be accomplished by looking for entries that begin with a sigma 0xe5. When a file or directory is deleted under a FAT filesystem, the first character of its name is changed to sigma. The remainder of the directory entry information remains intact. "

Click to expand...

Yes, I do not disagree with this part. However entries in the file allocation table are 'reset'.

Need to read the entire part you link to:

Recovering directory entries from FAT filesystems as part of recovering deleted data can be accomplished by looking for entries that begin with a sigma 0xe5. When a file or directory is deleted under a FAT filesystem, the first character of its name is changed to sigma. The remainder of the directory entry information remains intact.

The pointers are also changed to zero for each cluster used by the file. Recovery tools look at the FAT to find the entry for the file. The location of the starting cluster will still be in the directory file. It is not deleted or modified. The tool will go straight to that cluster and try to recover the file using the file size to determine the number of clusters to recover. Some

mdmarqphoto said:

https://www.minitool.com/data-recovery/fat32-data-recovery.html

"Here is how a file is deleted under Windows OS (operating system):

As a result, under the condition that new file is not saved to the place where deleted file was located, you can recover deleted files from FAT32 drive correctly by modifying corresponding location and searching for E5 deletion mark according to data storage structure and data link of deleted file ."

Click to expand...

The latter is incorrect, plain and simple. You would need to edit the file allocation tables too.

--
Joep

I've been playing with SD and MicroSD cards for years (albeit not in the capacity that many here do) and I've only had a few card corruptions. Most of the time was when I was young and dumb with the cards and didn't understand yet to safely eject them. Once was with a very old card that went unused for a while. Once was with a poorly formatted card. The last time I had a SD card corruption was the MicroSD card from my phone when I used my phone to take pictures at Niagra falls without a water resistant case (it had a case but the phone and case are not water resistant). I've never had any trouble with corrupt SD cards other than those times. And even the last time was just because I exposed it to conditions it wasn't meant to be in.

With file organizing, I've also never had problems. Windows explorer is just fine. It helps that I use a very fast SSD to keep all my files on. Honestly, I don't see the point that you're trying to make.

There's a simple solution for what it seems like you're trying to suggest: Select the image->Press the DEL button.

Plus, changing the first letter of a file name shouldn't do any harm to the card. If it did then there's something very wrong with your computer or your card. That, or your card is junk and you should stop buying that brand.

--

It's a pretty simple task of 3D modeling something. It's getting it to fit that takes most of the time!