A new method called SpectralCanary allows for the auditing of text-to-image models to detect if they were trained on specific datasets. The technique embeds a subtle, low-frequency signal into images, which can then be detected in the generated outputs of a suspect model.
SpectralCanary offers a way to trace the use of image collections in the training of text-to-image (T2I) models. Researchers at the University of Maryland developed the system, which embeds a fixed "canary" image's low-frequency components into each image of a dataset before its release. This process leaves the original images visually intact and their captions unchanged. When a suspect T2I model generates images, a lightweight verifier can detect the presence of this embedded signal, providing evidence of the dataset's use.
The method aims to address the difficulty of proving unauthorized dataset usage, as models can learn collection-level characteristics without directly reproducing training images. Existing auditing techniques often struggle with a trade-off between verifiability, image fidelity, robustness, and ease of deployment. SpectralCanary's approach prioritizes these aspects by requiring only a single, deterministic transformation and no per-image optimization or auxiliary generation tools.
In tests across various datasets and model architectures, SpectralCanary demonstrated high accuracy in detecting the use of protected data. On the Pokemon dataset, it achieved an average per-generation true positive rate of 91.1% at a 1% false positive rate. When using a fixed threshold for model-level verification with 30 generations, it reached an accuracy of 94.8%. The system also performed well across artistic, medical, satellite, and fashion datasets when tested with Stable Diffusion 3.5, averaging 92.4% true positive rate while maintaining image quality, evidenced by minimal changes in structural similarity (SSIM of 0.989) and perceptual similarity (LPIPS of 0.022), and a near-zero change in Frechet Inception Distance (FID) of 0.05.
The researchers found that embedding the signal in the low-frequency spectrum of images is more reliable for transfer during model fine-tuning compared to mid or high frequencies. This is because low frequencies capture broad spatial structures that neural networks tend to learn and that are important for T2I generation.
Furthermore, SpectralCanary appears to be robust against adaptive attacks designed to remove the embedded signal. The study indicated that aggressive modifications to remove the signal resulted in a significant degradation of image quality, with over 25 points of change in FID.
The development of SpectralCanary comes at a time when the use of large datasets for training AI models is common, raising concerns about copyright and data provenance. Proactive dataset-use auditing, as implemented by SpectralCanary, offers a mechanism for rights holders to verify how their data is being utilized without preventing its legitimate use. Previous research in data auditing has explored various methods, including embedding marks or watermarks into datasets, but these often face challenges in maintaining image quality, generality across different model types, or require extensive control over the training process. SpectralCanary's simplicity and effectiveness in preserving image fidelity while enabling detection make it a notable advancement in this field.