User:Archivist127/archive/file manager bug

A possible bug in file managers causes files being moved from the source partition to be erased without being transferred properly due to a malfunctioning check for the existence of a target file or folder.

A possible cause is that the code responsible for deleting an incompletely moved target file is inadvertently executed for the source file or folder, though that would not explain why it only happens with moving, not copying.

Exhausted storage
On an ordinary morning, a moment of impatience triggered a software bug lead to data loss. I was about to take a photo using my own phone. When I opened the camera, the annoying message informing that the internal storage is exhausted appeared. For those interested, the package name of the pre-included camera software is.

The message is phrased with at least ten words, none of which are worth reading. That box with the letters in it at the corner the field of view looks unpleasant enough. I was not prepared for this, as the camera software does not show the approximate count of remaining photos, like many other phones do, to warn the user in advance.

Because it was the internal storage, it being full may lead to stability issues of unrelated apps, so I needed to clear out the internal storage as soon as possible.

I realized that the MicroSD card still had around half a gigabyte free. What I should have done is just switch to the memory card and take the photo there, and then calmly move the photos to a hard drive using a computer, using MTP (which is more stable and loads faster when disabling thumbnails), or FTP. I was not able to act patiently and rationally. I just wanted this parasitic ENOSPC message, which almost appeared insulting, out of my face as quickly as possible.

I fired up the file manager. I was unable to use ES File Explorer (a 2015, pre-adware version of it) due to Google's restrictions of storage access under the pretext of security without providing options, meaning they succumbed to apps which misused file access. But it could be part of their business plan of promoting their Google Drive cloud storage. But, as known, memory cards are better at privacy, speed, and user control.

My plan was moving a small part of the >10GB camera folder to the free half-gigabyte of the memory card, and abort the move shortly before it is filled, to free up few hundred megabytes from the internal storage. I selected the folder using the pre-included file manager (which can write to MicroSD). For those interested, its package name is. But then, that file manager refused to move the files due to the total size being above 4 GB. This limitation was implemented into that file manager presumably due to FAT 32 being popular for memory cards. However:


 * The MicroSD card was formatted in the ExFAT file system, not FAT32, which means file sizes are not limited to 4 GB, but, as far as I know, multiple exabytes.
 * The total folder size was above 4 GB, no individual file. Yet the file manager was programmed to refuse moving selections of above 4 GB. It appears that the software was not properly tested before being rolled out.
 * I could have selected individual files, but as pretty much every file manager except ES File Explorer lacks range selection, where all items between two highlighted ones can be selected using range selection, I would have to tap dozens of files individually. No, thanks. Maybe I should just have sorted the list of files by size, but under my frustration, I was not able to act that patiently.

I could also have moved the files using ES File Explorer to, which it can write to, but due to my furiousity, I just seeked out the fastest viable solution. And Google's proprietary "Storage Access Framework" is buggy garbage anyway.

The disaster
I remembered that Android has an integrated file manager. It is codenamed DocumentsUI (package name: ), and is accessible from the storage settings. I opened it, selected the camera folder, the target folder on the SD card, and started moving. Shortly before the space on the SD card was exhausted (around 100 MB free), I tapped cancel on the push notification which shows the file transfer progress.

Now I wanted to check the free storage of the internal storage, and it read 10.8 GB. So much free space in few seconds couldn't possibly mean any good. Shortly after, I realized that the camera folder was entirely gone.

Anyone who had any large-scale data loss will know the pain I am speaking of. And this was internal storage, so data recovery software that uses mass storage access is powerless here, and the TRIM command has possibly destroyed any hope of recovering it. Even with mass storage access, the internal storage is encrypted. The mobile phone's internal storage is basically a reliable file shredder. Or, dare I say, write-only memory. The only thing I could have done is turn off the phone immediately, find a way to become a millionaire so I can afford a forensic laboratory to recover part of the data after years, and not use the phone until then. Sorry, but that appears to difficult, and the chances of success are not even certain.

Sure, I do take backups as soon as the files are on the external hard drive. But getting them there from the phone is a form of art. Existing methods of file transfer like Media Transfer Protocol (MTP) are inconvenient and/or unreliable. FileZilla can not move files, only copy, meaning I have to delete carefully to avoid deleting files not transferred; and Nemo file manager does not retain the original date and time.

Now people will suggest to copy, verify and then delete files from the source instead of direct moving. With that, there is a problem, however: In order to make space free from the source device or partition, one would have to select and delete the exact same files that were transfered afterwards, without accidentally selecting anything that was not copied, which takes emmense effort. No, thanks. (Well, in this case, the entire thing is gone anyway, but that was not planned.) But that would have been too too stressful and have required lots of effort, which was not viable under that momentary lack of patience. Even with patience, that would have required a lot of time. So, impractical.

I could have used USB-OTG, but somehow, every time I use that, the phone slows down significantly during transfer, becoming near-unuseable from the lags, making it impractical as well. And files' date and time information is discarded anyway.

Honestly, file transfer is already one of the least funny activities in everyday life. I select lower photo resolutions and purchase larger memory cards not mainly because I need more space, but to postpone file transfers for as long as possible. But then, such a thing happens, making me dislike it even more.

Reassessment
I later reproduced this bug and found out that this happens every time with Google's pre-included DocumentsUI file manager. If a folder is being moved, but the move is aborted thorough that push notification, the source folder is deleted. If I ended the file manager using force quit, it would observably not have happened.

I tested this behaviour on other file managers (pre-included MediaTek file manager, ES File Explorer, Amaze File Manager, RhythmSoft File Manager HD, OI File Manager, Samsung "My Files", Windows Explorer, Nemo for Linux, and probably a few more). None of them had this bug.

Except Google's "DocumentsUI" file manager. Any other tested file mangager would not have destroyed thousands of photos and hundreds of videos.

How is a user supposed to foresee this? One expects a "cancel" button to do just what it says. Cancel should mean cancel. Cease all file actions. Do nothing anymore. Period.

But instead, the true meaning of that button in Google's file manager is "Cancel and delete current item from source.", or metaphorically "cancer", as it "eats" data away.

So many co-incidences lead to this disaster. Had one of these things been different, such as MediaTek's file manager not refusing to transfer an selection of files with a total size of above 4 GB, the loss of thousands of photos and hundreds of videos would not have occured. Or had it been stored on the MicroSD card instead of the less controllable internal storage, forensic data recovery software could have externally rescued a lot of it.

MediaTek failed, and Google failed harder.