Well, the fix turned out to be really simple. Sure, it was a rewrite, but much better. Basically, I just read all of the virtusertable into a hash where
$virt{$TO} = $FROM;Then I read in the 1,000+ entries into the same hash. Invalid entries were automatically corrected, duplicates had no effect, and additions were simply added.
Very simple. Very clean.