From the man page:
You must compile libcaca package with support of Imlib2 to be able to
load a wide variety of image formats. Otherwise you will only be able
to load regular BMP files.
Checking libcaca’s template there’s no imlib2. If you really need it you can add and compile it with xbps-src yourself. I opened a PR to add imlib2 to libcaca. Now let’s wait and see