Home | News | Download | Packages | Forum | Wiki | Github

Which tar format?


#1

I was asked to add Void Linux to the list of OSes available to install via PINN (a fork of NOOBS) on the Raspberry Pi.
As you have a .tar.xz install format I thought it would be easy, but since PINN/NOOBS uses bsdtar it failed with several of the following warnings (which were treated as errors):

bsdtar: Pathname can’t be converted from UTF-8 to current locale.

Could I ask which tar tool you have used to archive the OS installation and which tar format it uses (pax, gnu, ustar etc)?
Are there any requirements to keep this format, or could it be changed to one compatible with NOOBS/PINN?


Raspberry Pi Zero W firmware
#3

Xz is not the issue. It is a question of the tar fornat used before it is compressed with xz.


#4

OK @procount I tried with this XZ archive in the repository: void-rpi-rootfs-20160420.tar.xz

$ unxz void-rpi-rootfs-20160420.tar.xz
$ file void-rpi-rootfs-20160420.tar
void-rpi-rootfs-20160420.tar: POSIX tar archive

#5

Interesting that it is already in posix format as it was suggested to me that “locale=POSIX tar void…” when creating the archive would force it to use posix format and therefore use binary format for the filenames. When I looked in the tar file I saw it used pax headers and ustar was mentioned so I wasnt sure what format it was. (I’m no expert in tar formats just trying to get it to untar without error messages)


#6

What’s new @procount ? Have you made any progress with your tar issue ? :compression:


#7

Not yet as I’ve been too busy. Are you a Void maintainer?

Some history…
NOOBS originally used tar to install OSes.
Arch linux decided Arch must use bsdtar so was excluded.
My fork of NOOBS called PINN changed to bsdtar to reinclude Arch and it worked with all the other OSes too.
NOOBS now uses bsdtar too.

Whilst I could make an exception in PINN to go back to tar, I don’t think NOOBS would.

So I either need to confirm the bsdtar error messages are just warnings and I ignore them or Void linux needs to be tar’d on a manner that bsdtar can untar it without errors in a non-locale environment


#8

Not at all !! :laughing: …sorry, I’m just a passionate user in love with Void Linux. :wink:

As I know absolutely nothing about PINN & NOOBS, I can’t answer you if you should or should not make an exception for Void.


#9

This is clearly a problem on your side, with your locale.
I just tested it and bsdtar extracts the latest rootfs tarballs without problems.

They are created with gnu tar and the --posix flag.


The current tarballs are created without --xattrs, which shouldnt make any difference in this case.


#10

Thanks for the detailed info.
Noobs/pinn run in a simple buildroot uClib env with no locale info built in.


#11

With LANG=en_US.ISO-8859-1 bsdtar -vxf ../void-rpi-musl-rootfs-20160420.tar.xz and a bit filtering i was able to get a list of files that cause this problem.
Arch ships the same files, but I have no idea how they create the tarballs.

bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./etc/ssl/certs/T \234RKTRUST_Elektronik_Sertifika_Hizmet_Sa \237layıcısı_H6.pem
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./etc/ssl/certs/T \234RKTRUST_Elektronik_Sertifika_Hizmet_Sa \237layıcısı_H5.pem
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./etc/ssl/certs/T \234BİTAK_UEKAE_Kök_Sertifika_Hizmet_Sa \237layıcısı_-_Sürüm_3.pem
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./etc/ssl/certs/NetLock_Arany_=Class_Gold=_F \221tanúsítvány.pem
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./etc/ssl/certs/EBG_Elektronik_Sertifika_Hizmet_Sa \237layıcısı.pem
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./usr/share/ca-certificates/mozilla/T \234RKTRUST_Elektronik_Sertifika_Hizmet_Sa \237layıcısı_H6.crt
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./usr/share/ca-certificates/mozilla/NetLock_Arany_=Class_Gold=_F \221tanúsítvány.crt
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./usr/share/ca-certificates/mozilla/T \234RKTRUST_Elektronik_Sertifika_Hizmet_Sa \237layıcısı_H5.crt
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./usr/share/ca-certificates/mozilla/T \234BİTAK_UEKAE_Kök_Sertifika_Hizmet_Sa \237layıcısı_-_Sürüm_3.crt
bsdtar: Pathname can't be converted from UTF-8 to current locale.
x ./usr/share/ca-certificates/mozilla/EBG_Elektronik_Sertifika_Hizmet_Sa \237layıcısı.crt
bsdtar: Error exit delayed from previous errors.

#13

I extracted using busybox tar which resulted in no error messages.
Using bsdtar I got the afore mentioned errors.
HOWEVER, when I diffed the 2 extracted trees with each other, it was the TAR extracted tree that was missing 4 files compared to the bsdtar extracted tree:

Only in bsdtar: etc/ssl/certs/1874d4aa.0
Only in bsdtar: etc/ssl/certs/65b876bd.0
Only in bsdtar: etc/ssl/certs/China_Internet_Network_Information_Center_EV_Certificates_Root.pem
Only in bsdtar: etc/ssl/certs/TÜBİTAK_UEKAE_Kök_Sertifika_Hizmet_Sağlayıcısı_-_Sürüm_3.pem

Which suggests that bsdtar extracted the files successfully despite the error messages. But the error messages cause NOOBS/PINN to abort.


#15

I compared the Arch tar file with the void tar file:

head -c 2000 ArchLinuxARM-rpi-latest.tar
./bin000777 000000 000000 00000000000 13023007013 013152 2usr/binustar00rootroot000000 000000 ./boot/000755 000000 000000 00000000000 13032257657 012146 5ustar00rootroot000000 000000 ./boot/fixup_x.dat000644 000000 000000 00000023023 13032045471 014307 0ustar00rootroot000000 000000

head -c 2000 void-rpi2-rootfs-20160420.tar
./PaxHeaders.31471/.0000644000000000000000000000013212705710611011027 xustar0030 mtime=1461162377.248646506
30 atime=1461162372.681647024
30 ctime=1461162377.248646506
./0000755000000000000000000000000012705710611011211 5ustar00rootroot00000000000000./PaxHeaders.31471/lib0000644000000000000000000000013212705710611011440 xustar

So it looks to me like the Arch file omits the PAX headers.


#16

@procount This morning I found this interesting website about tar/pax:

http://invisible-island.net/autoconf/portability-tar.html


#17

I have now created a draft installation of VoidLinux that can be installed with PINN (my fork of NOOBS for the Raspberry Pi).
It will automatically select the correct version according to the model of PI, yet give you the choice of installing the standard rootfs or the musl one.
I created three slideshow pictures (taken from the Void website) that are displayed as it is installed. I need to reduce their size a bit so they will fit on the touchscreen display, but if anyone has any better ones, please feel free to provide them.
I would welcome any feedback or comments.

PINN can be downloaded as pinn-lite.zip from https://sourceforge.net/projects/pinn/files/
Documentation can be found at https://github.com/procount/pinn/blob/master/README_PINN.md
Format an SD card as FAT32 and unzip pinn-lite.zip onto it.

To get the tar files to work with PINN, I repackaged them with bsdtar but using --format=“gnutar”. I don’t know if this has had any effect, but it still seems to work ok from my limited testing.