Home | News | Download | Packages | Forum | Wiki | Github

[SOLVED] Really bad performance when copying large files to FAT32 SD Card


Hello to the Void!

Because of a hobby of mine, i often need to drop big files (2-3,5GB) onto SD cards (which have one FAT32 formatted partition on them).
On other distros, the speed was always displayed as a couple of 100 megs per second, until the file was completely “copied over”, then it dropped to 0 and it took another 5-10 minutes to complete the copy. It was like this because of the SD card’s cache, so it was pretty normal behaviour.
On Void however, the copy starts at the normal speed, then drops by an EXTREME immediately, dangling around at 20-200KiB/s. Because it sometimes stops completely, i have even switched my OS to FreeBSD before because i wanted to get it done, but now i want to fix it on Void.

Where could that issue come from?


How do you mount your SD card? What are your mount options?

How do you test the speed? Have you tried using dd?


I mount it using the KDE I/O interface (kio), meaning through the external devices menu in KDE.
Mount options: /dev/mmcblk0p1 on /run/media/jonas/43FC-9668 type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
I can see the speed in the copy status dialog. I have not yet tried using dd (what options would you recommend there?)

I also just checked dmesg, and there’s a shitton of messages like this: [29131.122995] mmcblk0: error -84 transferring data, sector 7206664, nr 8, cmd response 0x900, card status 0xc00
I’ll try figuring out what it’s trying to tell me.


Yeah, these dmesg messages look bad, but aside from that I noticed that you don’t use the async mount option. It helps with the speed, sometimes.

About dd: just use it to copy some file to check the speed.

dd if=~/somefile of=/run/media/jones/43FC-9668/somefile


That probably means your FAT32 partition is in bad shape? :thinking:

$ man fsck.fat


If switching OS fixed file transfer, the problem is the destination. Is your Void disk full? What fs does it use? the problem has nil to do with your particular distro. It’s about hardware and filesystems.

If you’re switching OS on whims like that, reformat the SD card with UDF unless your scenario is a camera with FAT32. UDF is native to BSDs but works on all OSes. Do a backup and buy a new SD card. They have a finite life span.


Is UDF an improvement over FAT32? I’ve never tried it myself. But I noticed following the UDF format link from the link above it said Maximum file size 2 GB for FAT32, yet I just recently copied a ~5GB zip file from one Windows XP PC to another with an FAT32 USB stick. So is that page misleading in other ways? Also UDF looks mainly intended for optical drives:
"UDF is developed and maintained by the Optical Storage Technology Association (OSTA)."
Anyway the real information I have to pass on here is some machines have a BIOS option to change the USB transfer speed. Probably it is irrelevant to the above problem but might be worth checking.
Mounting and unmounting from the command line would make it easier to try other mount options for testing, this is how you would mount with default options then unmount it, umount syncs too, so when the command completes and the cursor returns you know it is safe to remove the drive (well hopefully :giraffe: ) :

# mount /dev/mmcblk0p1 /mnt
# umount /mnt


The comparison chart copied from Wikipedia in 2015, it claims. If there’s a chart error it’s only off by 2 GiB. Limits are a function of which FAT is under discussion and how it’s tuned. The chart’s discussion is tuned for max cross-OS interop, not theoretical maximums. You’re likely using exFAT. The chart footnote says why he didn’t pick exFAT.

Why UDF? The OP mentioned BSD on which UDF is native. Yes UDF beats XXXFATNNN on BSD, as any Linux filesystem beats it on Linux. I just proposed UDF for someone who flips between Linux and BSD, which doesn’t support Linux filesystems. Yet UDF can also be your universal filesystem across Mac/Windows, not just Linux/BSD. Indeed it’s the only competitor to XXXFATNNN for such a use case.

The flocking instinct, inertia, and Microsoft have more to do with market penetration than technicals. You still find new embedded systems running DOS these days. Common doesn’t mean better.


@mcz The dd command “worked”, it “copied” the file in about 15 seconds (of course, it wasn’t actually finished, and running sync would make all those errors appear again. EDIT: Without every finishing the sync, of course.)

@cr6 fsck only reports errors after a failed file transfer, when i delete the file and let fsck apply fixes, there are no more errors. I did that multiple times.

@nixit The destination is the SD card. The file is coming from my home partition which has 215GiB of free space (this is a new SSD) and going onto an SD card with 16 or 32GB total space, and there were 12GB of free space on it when i last tried this.
I didn’t exactly switch to FreeBSD, it was only a temporary solution, i ended up switching to Artix back then. And other file systems are not an option, as this SD card is used with a 3DS. Also: The SD card is pretty much new as well.
This problem has yet to occur on any other system than Void - everything worked fine on Manjaro, Artix, Gentoo, FreeBSD, Windows, and Debian. I used an old Manjaro Architect image to copy the file this time, but that’s a major inconvenience and i would really like to get it working on Void.

I have tried manual mounting at this point, exactly the same way i did it in Manjaro, and it didn’t work.


You’re copying from a very fast to a relatively slow medium. And perhaps you have a lot of RAM? You can try to adjust the dirty_bytes, see e.g. here and here.


The wikipedia page I linked to suggests UDF doesn’t work on BSD, but from your info it does. So perhaps that is the source of the contradictory information on UDF, and if you have been using it successfully then you are no doubt correct.

Anyway I was just looking at the udisks template, as I remembered how udisks1 would set firewire transfers to the slowest speed as speed detection wasn’t working then. But it turns out udisks2 has changed it’s release repo address, and it looks like the Void template is pulling from the old one not the latest, as the udisks2 version in Void is from 2016.
This page shows the 2 repos now available:
So perhaps that could explain the difference with other distros, if they are using a later version.

(Edmond Dantes ) #12

FreeBSD has UDF2 support, but it’s not enabled by default, you need to add the module to your kernel command line configuration; so, in /boot/loader.conf :


Also, please note that there’s no BSD, they’re all very different OSs, have different kernels,userland, filesystems, init, configuration files and syntax, different hardware support and software availability.

Regarding the possibility of using dd as a file transfer rate tester, I agree with @mcz, there’s no better tool for this. Personally, in these cases I like to pipe dd through pv(1) (not in base, package is available in repo), thus to to show a dynamic progress bar and get some nice info about the tranfer rate. For example:

 doas dd if=void-rpi3-musl-20171007.img.xz bs=1M conv=sync,noerror,notrunc | pv -s 68M | doas of=/dev/sdb

Will return something like that:

67+1 records in [39.6MiB/s] [=========================================>                       ] 61% ETA 0:00:00


15 Biggest Wikipedia Blunders

Of course there are many BSDs, and many Linuces. Suffice to say “UDF on a thumb drive is simple and not a bad option.” Which the OP can’t use anyway, apparently. I guess the OP has this thing?

I think all he needs are mount opts in udisks2 configs or fstab or udev; trim/discard, flush, and atime issues come to mind. It’s easy to make one distro work like another:

  1. Mount the SD card in any distro X that “works.”
  2. Enter the mount command.
  3. Note the flags used by distro X to mount the SD card.
  4. Install those flags in Void.


@hralgmir the older udisks version in use would really explain a lot. Maybe i can create a PR to update it.

@Montecristo i personally just use status=progress on the dd command to watch my progress, but that does look very nice!

@nixit My hobby is modding 3DS games (replacing and editing textures and editing in-game dialog, mostly), and the games i edit are very big ones, which then need to land on the SD card in .cia format in order to be installed (to the same SD card). A lot of system software (custom firmware stuff) is also on the SD card, so it’s essential (and, unfortunately, it has to be FAT32).
I’m gonna try copying those mount flags over the next days and will report back!


Now the Void devs are aware of this problem.
I told them on IRC, as soon as I saw @hralgmir 's comment. :wink:

EDIT: …and as you can see here it’s not so easy! :scorpion:


My UDF buzz was aimed at others. UDF is more useful if a choice exists. You didn’t say otherwise. Failing to spec hardware and usage scenario up front is like saying “My email is broken.”

Try the mount opts. If your post nailed a udisks2 build bug in Void, then it was a good catch, but should have been a github issue. Devs don’t read this forum, it’s just a user ghetto they visit only to police.


@cr6 that’s good! I don’t have to do anything then, lol.

@nixit this is actually the first time i’ve heard anyone talking about UDF. I vaguely remember Windows offering me a way to format my CDs with UDF, that’s it. I’m gonna read a bit about its history as it sounds very interesting for external drives in general, if it has some stuff that FAT32 (and NTFS, which i still use for interoperability) doesn’t have.
I will try the mount options as soon as i have time for it.
And i think i’ll continue writing to the forum first, just to see if someone had the same issue, as it might always just be a configuration issue. When i find something that is, without doubt, a packaging issue, i’m gonna file a GitHhub issue.


@Follpvosten Out of curiosity: could you try to mount your SD card by using pmount (Permits normal users to mount removable devices), and run another :stopwatch: speed test?


(Adding more s/w is bad debugging practice)


:joy: lol

$ sudo xbps-install pmount

Name   Action    Version           New version            Download size
pmount install   -                 0.9.23_6               65KB 

Size to download:               65KB
Size required on disk:         436KB
Free space on disk:           6228MB

Do you want to continue? [Y/n]

Seriously… 65KB to download, it’s nothing compared to the enormous KDE! :sauropod: