BasicImporter

Overview

BasicImporter is an Unreal Engine plugin (UE 4.26–5.7) that solves the fragmented data-import problem: no single, unified path exists for bringing external files — JSON, CSV, XML, or custom formats — into engine-usable data. The plugin builds a four-stage pipeline (file I/O → parser → universal intermediate representation → consumer) that works the same way regardless of format. It supports both editor-persistent UDataAssets saved to disk and runtime-transient containers for modding or live config.

Key Features

• Factory/registry architecture — adding a new format requires one parser class and one registration call; zero changes to existing plugin code
• Async chunked pipeline — 64 KB streaming reads on a worker thread with game-thread callback marshaling; imports 50 MB+ files without frame hitches
• Universal intermediate representation (FBIDataNode) — decouples parsing from consumption; all parsers produce the same tree, all consumers read from it
• Blueprint-callable API — UGameInstanceSubsystem exposes sync and async import, dot-path value access, and DataTable population to Blueprint
• Auto-chunking & FlatData cap — large assets are automatically split into chunk sub-assets; the FlatData TMap is skipped above 50 K entries to prevent editor Details Panel freezes
• Cross-version compatibility — BIVersionCompat.h macros abstract UE4/UE5 API differences (SavePackage vs Save, FTopLevelAssetPath, etc.)

Technical Decisions

The central design decision is the FBIDataNode intermediate representation — a pattern borrowed from compiler IR design. Every parser converts raw bytes into the same recursive variant tree; every consumer reads from that tree. This cleanly separates “how to parse format X” from “what to do with the data,” enabling the registry’s open/closed extensibility. The async pipeline uses Async(EAsyncExecution::TaskGraph, ...) rather than UE5-only UE::Tasks::Launch to preserve backward compatibility back to UE 4.26. Memory is kept bounded through 64 KB chunked I/O with a single reusable buffer and UE-native containers exclusively (TArray, TMap, FString — no std:: types).