"Low Level Format" vs. "Format" on Canon

Started Aug 17, 2009 | Discussions
RUcrAZ
Veteran MemberPosts: 4,162
Like?
"Low Level Format" vs. "Format" on Canon
Aug 17, 2009

Hi,

To do "bulk erasing" I've been Formatting various brands of SD cards for Canons, but always wondered what does "Low Level Format" do, what are its benefits, downsides, and what additional setup steps may be needed after the Low Level Format.

(I've low-level formatted HDs for PC, but that was to start a-fresh and reinstall everything situation.)

So my question is not the esoteric one, but: What do I have to do differently after low-level Formatting, that I am not doing after simple Formatting?
Thank you
RUcrAZ
Canon A620, Canon SX10 IS

jwampole
Junior MemberPosts: 34
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to RUcrAZ, Aug 17, 2009

Hello all,

According to the G10 manual and the Magic Lantern G10 book low level formatting should be done if you suspect the recording speed has slowed down. The books both mention that a low level format will take 2-3 minutes. Neither book says anything about any additional steps following low level format.

Personally I've never done a low level format, but I may just to see what happens. Relatively recently I began formating the card every time I downloaded images to the computer. I didn't do this with my earlier digital cameras, and there were never any issues. However, I've read some horror stories about not formatting the card. As a safety factor I format the card in the camera, not while attached to the computer.
--
It's a big world, and I want to see it all!

Reply   Reply with quote   Complain
AdamT
Forum ProPosts: 56,410Gear list
Like?
The Difference ---->
In reply to RUcrAZ, Aug 17, 2009

Basically a standard format just deletes the file allocation table and the images on the card can be recovered using a recovery tool - I'd use this method most of the time, its faster and if you lose one and not used the card afterwards, you can usually recover the data witha simple photo recovery program..

Low Level format fills all the space on the card with Zeros so if you sell the card with the camera, the buyer can't use "photo-recovery" to find all the cheeky pics of the missus in the bath ..

-- hide signature --

Please ignore the Typos, I'm the world's worst Typist

 AdamT's gear list:AdamT's gear list
Sigma DP1 Fujifilm X-S1 Canon PowerShot G1 X Nikon Coolpix P330 Olympus OM-D E-M5 +1 more
Reply   Reply with quote   Complain
glenncombs
Senior MemberPosts: 2,055
Like?
Another Low Level Format Use
In reply to AdamT, Aug 18, 2009

Low level formatting is also handy to make memory cards work in a camera where they fit the specifications, but are not recognized by the hardware. Sometimes with Canon cameras, a new card inserted with display No Memory Card. Low level formatting the card in the camera will straighten out the problem, if indeed it fits the specifications for the camera. I've found this to be true for certain brands of 4 GB SD cards in my S2. After the low level formatting, they work fine. As stated in the earlier post, further low level formatting is not usually necessary, unless you want to completely wipe all data off of the card.

Glenn
--
Alexander...our wild child

Canon S2, A610, SD880

Reply   Reply with quote   Complain
RUcrAZ
Veteran MemberPosts: 4,162
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to RUcrAZ, Aug 18, 2009

Thank you all very much for your prompt and clear explanations. It is appreciated.

Reply   Reply with quote   Complain
Wayne Larmon
Veteran MemberPosts: 9,251
Like?
How does low level formatting speed up recording speed?
In reply to jwampole, Aug 18, 2009

jwampole wrote:

According to the G10 manual and the Magic Lantern G10 book low level formatting should be done if you suspect the recording speed has slowed down.

How does low level formatting speed up recording speed?

Wayne

Reply   Reply with quote   Complain
gkl
gkl
Senior MemberPosts: 2,079
Like?
Re: The Difference ---->
In reply to AdamT, Aug 18, 2009

AdamT wrote:

Low Level format fills all the space on the card with Zeros so if you sell the card with the camera, the buyer can't use "photo-recovery" to find all the cheeky pics of the missus in the bath ..

Something's wrong here. The G10 manual states that a low level format takes 2 or 3 minutes. If that's so, it would take over 22 minutes to fill an 8GB card with zeros, assuming the G10 can write 6MB sec (which may be optimistic). Either the manual is incorrect, or it's not erasing everything. cheers, gkl

Reply   Reply with quote   Complain
glenncombs
Senior MemberPosts: 2,055
Like?
Re: The Difference ---->
In reply to gkl, Aug 18, 2009

gkl wrote:

AdamT wrote:

Low Level format fills all the space on the card with Zeros so if you sell the card with the camera, the buyer can't use "photo-recovery" to find all the cheeky pics of the missus in the bath ..

Something's wrong here. The G10 manual states that a low level format takes 2 or 3 minutes. If that's so, it would take over 22 minutes to fill an 8GB card with zeros, assuming the G10 can write 6MB sec (which may be optimistic). Either the manual is incorrect, or it's not erasing everything. cheers, gkl

I don't know specifically about the G10 and formatting. But, I have the SD880, and I have low level formatted 32 GB SDHC Class 6 memory cards (Transcend), and it typically takes only around 10 seconds, perhaps. Minutes is definitely not the case, and I am talking about Low Level format, and not Format. It does not take very long to Low Level Format these memory cards. And yes, it does put zeros in all of the memory, erasing all data.

Glenn
--
Alexander...our wild child

Canon S2, A610, SD880

Reply   Reply with quote   Complain
gkl
gkl
Senior MemberPosts: 2,079
Like?
Re: The Difference ---->
In reply to glenncombs, Aug 19, 2009

glenncombs wrote:

I don't know specifically about the G10 and formatting. But, I have the SD880, and I have low level formatted 32 GB SDHC Class 6 memory cards (Transcend), and it typically takes only around 10 seconds, perhaps. Minutes is definitely not the case, and I am talking about Low Level format, and not Format. It does not take very long to Low Level Format these memory cards. And yes, it does put zeros in all of the memory, erasing all data.

In modern computing, low-level formatting is supposed to identify/flag bad blocks and (optionally) reinitialize all data areas on the device. These operations take a very long time on standard hard disks. It's not clear how such operations could be completed in several seconds over the much slower SD card interface. (Max write speed on the G10 is something like 6 to 9 MB per second.) Perhaps entire sectors can be reset to zero electronically - this would be faster than actually transferring zero data to all areas of the card. Still, I'd be wary that all of the data on a 32 GB card is actually being erased in 10 seconds. cheers, gkl

Reply   Reply with quote   Complain
AdamT
Forum ProPosts: 56,410Gear list
Like?
Re: The Difference ---->
In reply to gkl, Aug 19, 2009

(Max write speed on the G10 is something like 6 to 9 MB per second.) Perhaps entire sectors can be reset to zero electronically -

Whatever it does, nothing can be recovered afterwards

-- hide signature --

Please ignore the Typos, I'm the world's worst Typist

 AdamT's gear list:AdamT's gear list
Sigma DP1 Fujifilm X-S1 Canon PowerShot G1 X Nikon Coolpix P330 Olympus OM-D E-M5 +1 more
Reply   Reply with quote   Complain
JFScheck
Regular MemberPosts: 216Gear list
Like?
Re: The Difference ---->
In reply to AdamT, Aug 19, 2009

Not true - very easy to recover data from both hard drives and memory cards - even after a "low-level" format...

All the Low Level format does is check each block to ensure the block is still safe to write too vice just deleting the "index"...

I low level format all my memory cards once before I start using them in their respective cameras.

-- hide signature --

John 'JFScheck' Scheck
'Enjoy life. There's plenty of time to be dead.'

 JFScheck's gear list:JFScheck's gear list
Sony RX100 II Canon EOS 5D Mark III Sony Alpha 7 Canon EF 300mm f/4.0L IS USM Canon EF 16-35mm f/2.8L II USM +4 more
Reply   Reply with quote   Complain
AdamT
Forum ProPosts: 56,410Gear list
Like?
Re: The Difference ---->
In reply to JFScheck, Aug 19, 2009

Not true - very easy to recover data from both hard drives and memory cards - even after a "low-level" format...

I tried with Photorecovery and it found nothing which it did before.

-- hide signature --

Please ignore the Typos, I'm the world's worst Typist

 AdamT's gear list:AdamT's gear list
Sigma DP1 Fujifilm X-S1 Canon PowerShot G1 X Nikon Coolpix P330 Olympus OM-D E-M5 +1 more
Reply   Reply with quote   Complain
GreenPig
Regular MemberPosts: 214
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to RUcrAZ, Aug 19, 2009

My memory on this is a but fuzzy, but here's what I recall and my thoughts on formatting ...

I did some work on embedded micros many years ago (Windows CE) and we used Compact Flash cards for storing logged data. One probelm we faced was memory management of the card. We were logging hundreds of data items at 1min intervals and kept 1 month of data - managing the storage on the CF was tricky. Nowadays this is all handled internally on the card so the programmer shouldn't have to worry too much about it.

Back then (and I'm sure it is similar today), each sector of the card is treated as a common part - write to one byte and you write to the whole sector. Therefore the software will wait until it has a sector's worth of data before writing.

Now the tricky part - each sector has a life of approx 100,000 writes (numbers may be higher today). After that it starts to age quickly and may not work. In reality this can take a long time but in theory you need to be aware of it when managing the memory.

So, the memory management keeps a track of where it recorded the last sector and next time will write data to a different place - therefore evening out the wear across the whole card.

I'm not sure where it stores this index data but it just does, and it doesnt impact on the wear and tear.

I beleive the formatting of a card simply updates the index table for the data sectors.

Each file (image in our case) will use a number of contigous sectors. The first sector and sector count will be stored in the index.

When deleting files/images, first dector location in the index for those file is cleared (sector data is maintained) - effectively making those sectors free for writing in the future - but remember, we spread out the usage of sectors so it could be a long time before they see any action again - therefore recovery programs can repair the index data. Each card may use different algorythms so each manufacturer has there own recovery software.

When formatting, it may be just as simple as a delete all command.

A low level format may go deeper and check sector status and mark each one as clear or damagaed - but only in the index. Again the sector data may still be intact but recover is now much harder.

Bad sectors will cause a write to fail so the camera has to try the next available sector - and therefore slow down. Having the bad ones flagged (following a low-level format) will mean they are skipped and the speed is maintained.

I would doubt that anyone would see any real difference unless the card had lots of bad sectors.

Reply   Reply with quote   Complain
Freeman-Jo
Senior MemberPosts: 2,574Gear list
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to RUcrAZ, Aug 19, 2009

Just to get a info out there Canon in camera "Low Level Format" is no where near the real Low Level Format in the PC. Back in the 80s, when you do a "Low Level Format"(LLF) there is a good chance that if you do it incorrectly, you could destroy the drive to the point that the Operating System won't recognize it and unable to format. A LLF back then user have to give specific information about the drive including numbers of heads, sectors, tracks and interleaving sectors. As said, giving fault information can render your harddrive useless. Now a day, a true LLF is no longer required. However there is still format and quick format. Normal formatting will actually zero/delete/wipe out actual data on to every sector of the harddrive. Meaning chance of data recovery is very unlikely. A quick format is simply wipe out the allocation table and mark it look empty. An analogy for a quick format to a book would be a book w/ the table of content&index&page# rip out and replace w/ an empty ToC&index. In term of flash storage is a bit difference, as there is no physical interpretation like harddrive, there is no head, no track. If anything, it would be just sector. Thus a true LLF is no longer valid and not neccessary. Canon in camera LLF is actually more like a normal format, while non-LLF is just a quick format. There is a pro & con for doing a full format. As some have point out flash do have a limited write cycle, doing a full format mean writing to every area of the flash is not a good suggestion, at least not recommended to be done often. For flash device, such as Compact Flash(CF) or SecureDigital(SD), a quick format is simply enough for normal usage. There are time you should consider doing a full format, such as the first time you use the flash, after you recover a deleted file or corrupted, accidentally remove card(power down) while it was operating. As said, a full format will write to all area of the flash, which make it useful to diagnose if the flash have certain defective area, a full format will confirm that.

RUcrAZ wrote:

Hi,

To do "bulk erasing" I've been Formatting various brands of SD cards for Canons, but always wondered what does "Low Level Format" do, what are its benefits, downsides, and what additional setup steps may be needed after the Low Level Format.

(I've low-level formatted HDs for PC, but that was to start a-fresh and reinstall everything situation.)

So my question is not the esoteric one, but: What do I have to do differently after low-level Formatting, that I am not doing after simple Formatting?
Thank you
RUcrAZ
Canon A620, Canon SX10 IS

-- hide signature --

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
If picture worth a thousand words, how many megapixel is it?
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
http://www.jotographer.com/blog
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 Freeman-Jo's gear list:Freeman-Jo's gear list
Canon PowerShot G1 X Canon EOS-1D Mark III Canon EOS 5D Mark II Canon EOS 5D Mark III Canon EF 85mm f/1.8 USM +7 more
Reply   Reply with quote   Complain
gkl
gkl
Senior MemberPosts: 2,079
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to GreenPig, Aug 19, 2009

GreenPig wrote:

My memory on this is a but fuzzy, but here's what I recall and my thoughts on formatting ...

I did some work on embedded micros many years ago (Windows CE) and we used Compact Flash cards for storing logged data. One probelm we faced was memory management of the card. We were logging hundreds of data items at 1min intervals and kept 1 month of data - managing the storage on the CF was tricky. Nowadays this is all handled internally on the card so the programmer shouldn't have to worry too much about it.

CF cards have a built in controller whereas SD cards don't. This is a major difference. Still, the information you provided is useful. thanks, gkl

When formatting, it may be just as simple as a delete all command.

A low level format may go deeper and check sector status and mark each one as clear or damagaed - but only in the index. Again the sector data may still be intact but recover is now much harder.

Bad sectors will cause a write to fail so the camera has to try the next available sector - and therefore slow down. Having the bad ones flagged (following a low-level format) will mean they are skipped and the speed is maintained.

I would doubt that anyone would see any real difference unless the card had lots of bad sectors.

Reply   Reply with quote   Complain
gkl
gkl
Senior MemberPosts: 2,079
Like?
Re: The Difference ---->
In reply to AdamT, Aug 19, 2009

AdamT wrote:

I tried with Photorecovery and it found nothing which it did before.

This still doesn't guarantee the data has been overwritten by a "low level format", just that Photorecovery won't look for it. A knowledgeable hacker (or another program) may still be able to retrieve it. From experience with failed cards, I know some programs manage to (slowly) find things where others can't. Unfortunately, you have no way of knowing exactly what the camera vendor has implemented (flash memory specs are protected). The only way you can be sure your data is gone is to format and refill the card yourself with dummy data. cheers, gkl

Reply   Reply with quote   Complain
GreenPig
Regular MemberPosts: 214
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to RUcrAZ, Aug 27, 2009
Reply   Reply with quote   Complain
SBphoto 1
New MemberPosts: 9Gear list
Like?
Re: "Low Level Format" vs. "Format" on Canon
In reply to GreenPig, 1 month ago

I think that they might have leaned something about security. one can use a key to encrypt the SD card then a "low level format" dumps the key too. Without the key used for encryption the disk or sd card looks the same as if it had random data on it. I just remembered SSD's actually automatically encrypt the data as they write to nand actually.

some people in National security and other labs for example have recovered data after 7 passes with 0's. Unless you find a whole in the math. Encryption is the way to go. I read that  enterprises that use this method so it is easy to retire old disks shout the need for the 7 to 21 pass with 0's and random data.

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads