Skip to content

TPC TXI Parsing UTC list ordering Fixes#342

Open
vrifftech wants to merge 2 commits into
OpenKotOR:masterfrom
vrifftech:master
Open

TPC TXI Parsing UTC list ordering Fixes#342
vrifftech wants to merge 2 commits into
OpenKotOR:masterfrom
vrifftech:master

Conversation

@vrifftech

Copy link
Copy Markdown

Description

Fixes two PyKotor resource handling issues:

  1. Preserves original GFF list ordering when editing UTC feat and force power lists.
    Previously, removing one existing feat and adding a new feat could append the new feat to the end of FeatList, shifting later entries and causing TSLPatcher diff output to rewrite many unrelated list items. The updated behavior fills removed original list slots with newly added values before appending extras, producing minimal diffs for order-sensitive GFF lists.

  2. Improves embedded TXI parsing and roundtripping for TPC files.
    Some valid animated/cycle TPCs contain embedded TXI data after the packed image payload. The parser could miss or misinterpret this TXI data, especially for textures using proceduretype cycle, numx, and numy. The updated parser detects embedded TXI data more robustly, strips null terminators, and preserves original TXI text during roundtrip unless parsed TXI features are changed.

This fixes cases where tools using PyKotor could generate noisy or incompatible TSLPatcher instructions, and cases where embedded TXI metadata in valid TPC textures was not parsed correctly.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔧 Refactoring (no functional changes)
  • ⚡ Performance improvement
  • ✅ Test addition or update
  • 🔨 Build system or CI changes
  • 🎨 Code style changes (formatting, renaming, etc.)

Package(s) Affected

  • PyKotor (core library)
  • One or more tools under Tools/
  • Documentation
  • CI/CD
  • Other: ___________

Changes Made

  • Updated UTC dismantling/list serialization logic so ordered GFF lists preserve original slot positions where possible.
  • Changed feat/power list handling to fill removed original slots with newly added entries before appending additional new entries.
  • Prevented default-valued optional UTC root fields from being materialized when they were absent from the original resource.
  • Improved TPC embedded TXI detection so valid TXI footers can be found even when the texture layout depends on TXI animation metadata.
  • Added support for correctly parsing cycle/procedural TPC metadata such as:
    • proceduretype cycle
    • numx
    • numy
    • fps
    • blending additive
  • Stripped embedded null terminators when decoding TXI text.
  • Preserved original embedded TXI text during roundtrip when no parsed TXI features are modified.
  • Adjusted TPC writing for animated/cycle textures so the emitted header and embedded TXI layout are compatible with the expected game/resource format.
  • Added regression tests for UTC list slot preservation and optional field omission.
  • Added regression tests for embedded TXI parsing and roundtripping in TPC files.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this manually
  • I have tested on multiple Python versions (3.8+)
  • I have tested on multiple operating systems (if applicable)

Test Results

Targeted validation was performed for the UTC/GFF ordering fix:

- Editing g_bandon.utc by unchecking Guard Stance and checking Force Jump now produces the intended minimal TSLPatcher diff.
- The noisy output previously rewrote unrelated FeatList entries, KnownList0 entries, and default-valued root fields.
- The corrected output only changes the intended feat slot:
  FeatList\6\Feat=101

Targeted validation was performed for embedded TXI parsing in TPC files:

PLC_FrcDist01.tpc
- parsed_format: RGB
- animated: False
- layers: 1
- mips_per_layer: [1]
- embedded TXI roundtrip preservation: passed

PLC_FrcDist02.tpc
- parsed_format: DXT1
- animated: True
- layers: 4
- mips_per_layer: [8, 8, 8, 8]
- embedded TXI roundtrip preservation: passed

The PLC_FrcDist02.tpc embedded TXI was parsed as:

proceduretype cycle
numx 2
numy 2
fps 16
blending additive
downsamplemax 0
downsamplemin 0

Targeted PyKotor tests were run against the patched source. In the local validation container, the real kaitaistruct dependency was unavailable, so a minimal compatibility stub was used for the targeted test run. Full project test-suite execution should be performed in a normal development environment with project dependencies installed.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Message that will be displayed on users' first pull request

@bodencrouch

Copy link
Copy Markdown
Contributor

Thank you so much for putting effort into this @vrifftech I look forward to looking at the changes soon.
image

Thank you for doing your utmost to adhere to the overly meticulous pr template :D

@github-actions github-actions Bot added the package: pykotor Related to PyKotor core label Jul 6, 2026
try:
Tpc.from_bytes(data)
except kaitaistruct.KaitaiStructError:
except (kaitaistruct.KaitaiStructError, UnicodeDecodeError, OSError, ValueError):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh i'm glad you realized i needed the other exception checks as i'm sure i had them somewhere else in the codebase. I guess they never made it in after the Kaitai expansion fell in. I'm so interested if you ran into UnicodeDecodeError through guess-and-check or if you pulled from the previous exception gate though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

package: pykotor Related to PyKotor core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants