feat(ktx): add initial ktx input/output support.

[walcht]

Description

Fixes #2329.
Add initial KTX2 format support (closest similar plugin/format in OIIO is DDS).
(Note: 2nd attempt at opening this PR because the first one was signed off with
my other email).

Important

This is a very early draft PR because I have been trying since some days to fix
CI issues (compiles and passes tests fine on my Ubuntu machine but fails on
almost all CI jobs...). Any help with these CI issues is appreciated.

Consequently, there are a lot of TODO comments, and code snippets that haven't
been properly tested yet. All of these will be addressed after fixing CI issues.

This KTX plugin support obviously nulifies the benefits of using KTX in the
first place. That being said, this plugin is still useful so that end users
don't have to convert back and forth between KTX <-> supported format (e.g., PNG).
It is also useful to convert to and from KTX2 format using OIIO.

An example usecase would be Blender and its glTf import/export plugin.
(see KhronosGroup/glTF-Blender-IO#1896).

Ideally, at some point in the future, OIIO may introduce a new API to accomodate
texture formats that are mainly used for fast texture uploads to GPUs.

For details on KTX2 and what this PR supports and what it doesn't (alongside a
description of current limitations), see src/ktx.imageio/README.md (copied parts
of it at the end of this).

Tests:

Added initial testsuite but only for the input (i.e., info_command) part of the plugin.
Ideally, after fixing CI issues, the tests will be adjusted for ktxouput.

Checklist:

• I have read the guidelines on contributions and code review procedures.
• I have read the Policy on AI Coding Assistants
  => Haven't used any AI coding assistant tools in any capacity whatsoever.
• I have updated the documentation if my PR adds features or changes
  behavior.
  => Not yet. But will add docstrings alongside custom "ktx:" ImageSpec
  attribute descriptions.
• I am sure that this PR's changes are tested in the testsuite.
  => Partially. As stated above, I mainly opened the PR now in the hope of speeding
  up fixes for the CI issues.
• I have run and passed the testsuite in CI before submitting the
  PR, by pushing the changes to my fork and seeing that the automated CI
  passed there.
  => Only 2 jobs pass (clang-format, ABI check).
• My code follows the prevailing code style of this project and I
  fixed any problems reported by the clang-format CI test.
• If I added or modified a public C++ API call, I have also amended the
  corresponding Python bindings. If altering ImageBufAlgo functions, I also
  exposed the new functionality as oiiotool options.
  => No new API is introduced. This is just a plugin.

CI Issues:

1. latest version of libktx requires CMake >= 3.23 which causes failures in the CI.
   Tried adjusting the build_cmake script but still CMake 3.18.2 is used in some
   runners.
2. undefined symbol references to _ktxTexture (essentially, linking with libktx.a fails for
   some reason). This is probably due to me missing some crucial setup step (I thought
   LINK_LIBRARIES KTX::ktx is sufficient for linking to libktx to work - which, again, is on
   my machine).
3. failure to find KtxConfig.cmake on MacOS => Absolutely no idea why (passes in other
   Windows/Linux jobs).
4. IntelLLVM Not Supported => probably some libktx dependency causes this (there are
   some libktx build instruction details that I haven't looked at yet).

Feel free to edit this directly.

The sections below are copied from src/ktx.imageio/README.md. These will be updated
as this PR progresses.

Supported Encoders/Decoders

• Supported/Tested texture kinds:

  • SINGLE_TEXTURE_1D (TODO)
  • SINGLE_TEXTURE_2D
  • SINGLE_TEXTURE_3D (TODO)
  • CUBEMAP_TEXTURE (TODO)
  • ARRAY_TEXTURE_1D (TODO)
  • ARRAY_TEXTURE_2D (TODO)
  • ARRAY_TEXTURE_3D (not planned)
  • ARRAY_TEXTURE_CUBEMAP (not planned)

• Supported/Tested VkFormats:

  • VK_FORMAT_R8_UNORM
  • VK_FORMAT_R8G8_SRGB
  • VK_FORMAT_R8G8B8_SRGB
  • VK_FORMAT_R8G8B8A8_SRGB
  • VK_FORMAT_ETC2_R8G8B8_SRGB_BLOCK
  • VK_FORMAT_ETC2_R8G8B8A1_SRGB_BLOCK
  • VK_FORMAT_ETC2_R8G8B8A8_SRGB_BLOCK
  • VK_FORMAT_BC1_RGB_SRGB_BLOCK
  • VK_FORMAT_BC3_SRGB_BLOCK
  • VK_FORMAT_BC4_UNORM_BLOCK
  • VK_FORMAT_BC5_UNORM_BLOCK
  • VK_FORMAT_BC7_SRGB_BLOCK
  • VK_FORMAT_ASTC_4x4_SRGB_BLOCK

• BCn GPU block-compressed formats:

  • BC1 encoder/decoder
  • BC2 encoder/decoder (not implemented)
  • BC3 encoder/decoder
  • BC4 encoder/decoder
  • BC5 encoder/decoder
  • BC6HS/BC6HU encoder/decoder (partially implemented but not tested)
  • BC7 encoder/decoder

• ETC2 GPU block-compressed formats:

  • ETC2_RGB (aka ETC1) decoder
  • ETC2_RGB_A1 decoder
  • ETC2_RGBA decoder
  • ETC2 encoder (there are a few ETC2 encoders - including ConvectionKernels and ETCPACK)

• ASTC GPU block-compressed formats:

  • ASTC decoder (using libktx's ktxTexture2_DecodeAstc)
  • ASTC encoder (using libktx's ktxTexture2_CompressAstc)

• Basis Universal schemes:

  • UASTC encoder/decoder
  • ETC1S encoder/decoder

• Supercompression schemes:

  • ZLIB decompressor/compressor (using libktx's ktxTexture2_DeflateZLIB)
  • ZSTD decompressor/compressor (using libktx's ktxTexture2_DeflateZstd)

Limitations

• If original KTX2 format contained generated mip maps, there is simply no way
  to know which filter and its parameters that were used to regenerate these
  mipmaps. To avoid any issues, we simply early quit (return false) in open()
  if get_int_attribute("ktx:miplevels") > 1.
  TODO: mipmap generation using sane defaults (using OIIO).

• KTX2 supports many GPU-block-compression encoders and each one may have many
  different parameters that change the encoding quality (as usual, quality-speed
  tradeoff). There is simply no way to regenerate the exact same input texture
  without knowing these parameters and nor the KTX2 specs nor libktx nor
  KTX-Software tooling stores any (or sufficient) information about these params
  in the metadata.

• As stated in the comments in ktxinput.cpp, if given ktx texture is
  supercompressed then it has to be all decompressed (i.e., NOT the decompression
  of the underlying GPU texture format but rather the supercompression). This
  means that if you just need a particular subimage/miplevel, you pay the memory
  price of loading the whole KTX texture (which might be very large for 3D
  textures and texture arrays).

  • Per the specs:

  Discussion: Should each mip level be supercompressed independently or should
  the scheme, zlib, zstd, etc., be applied to all levels as a unit? The latter
  may result in slightly smaller size though that is unclear. However it would
  also mean levels could not be streamed or randomly accessed.

  Resolved: Yes. The benefits of streaming and random access outweigh what is
  expected to be a small increase in size.

• KTX2 writer writes the whole texture (i.e., all subimages/mipmaps) in the
  close() function (i.e., when the ImageOutput object is destroyed or requested
  to close). libktx does not provide a way to append or write subimages (is this
  problematic or contrary to the way OIIO expects us to write files?).

• KTX1 format is not yet supported. Adding support for it after finishing KTX2
  should be relatively straightforward (Note: KTX1 is officially deprecated and
  KTX-Software provides tools to convert from KTX1 to KTX2).

• Only LDR formats (to be more precise, only TypeDesc::UINT8). Adding support
  for HDR is straightforward (conversions for large number of enum values from
  VkFormat have to be written).

• bc7enc_rdo dependency for encoding/decoding BC1-7 formats does not (
  contrary to what the repo description suggests) support BC6HS/BC6HU HDR formats.
  See below on what we can use for BC6HU/BC6HS encoding/decoding.

Dependencies

• libktx: for general KTX@ format support (loading of KTX2 files, transcoding
  support, supercompression decompression support, etc.).

  • Commit hash: see OpenImageIO/src/cmake/build_Ktx.cmake
  • License: Many subresources. TODO: lib/etcdec.cxx's license is not open
    source but libktx exposes a function to decode ETC formats, do we use it?

• bc7enc_rdo: for BC1, BC2, BC3, BC4, BC5 and BC7 decoding/encoding.

  • Commit hash:

    dbe416d28a5530b4e8cc45b14bf034dc6b96bbde
    

  • License: MIT License
  • Note: I am working on a PR to push ktxTexture2_DecodeBcn function
    to KTX-Software (libktx) so that we no longer have to include this dependency here.

• etcdec: ETC2/EAC decoding.

  • Commit hash:

    972875d403ed8ac27e0f35c2f29d819e710a688a
    

  • License: MIT LICENSE
  • Note: libktx has etcunpack included by default. Maybe we can use it and
    remove this dependency?

Note: for BC6HS/BC6HU and ETC encoding support, we can use
ConvectionKernels. It is better to have this dependency built separately (i.e., not copied into
source ktx.imageio directory).

Personally, for ETC encoding, I would prefer to use ETCPACK since libktx already
uses its decoder and is the more standard choice (i.e., seems more official).
For BC6HU/BC6HS decoding, we can use the same dependency used by the DDS format
(bcdec.h). For encoding, we can extract the needed function from DirectX Texture
Library (MIT license).

Resources

• KTX2 Specs: https://registry.khronos.org/KTX/specs/2.0/ktxspec.v2.html
• Official Implementation (KTX-Software): https://github.com/KhronosGroup/KTX-Software
• Basis Universal Supercompression Implementation (used by libktx): https://github.com/BinomialLLC/basis_universal
• BC1-7 encoders/decoders with RDO: https://github.com/richgel999/bc7enc_rdo
• Comparing-BCn-texture-decoders: https://aras-p.info/blog/2022/06/23/Comparing-BCn-texture-decoders/
• ConvectionKernels: https://github.com/elasota/ConvectionKernels.git
• etcdec: https://github.com/iOrange/etcdec.git