I've written a couple of things to help with analysis of the word files, started a module with the relevant functions for organization purposes, etc., and at home I've left a test run going on a 1/11th subset of the whole word file.
The function I wrote takes two words, reverses the second one, looks for any number of instances of the second reversed word whole in the first, and then starts looking at the end of the first word for partial matches. I believe this handles the process of getting both the partials for each word in each other word. It's a headache, but I think it'll be doable. It returns a multidimensional hash of parts (which, now that my unbelievably lax mental QA process has actually thought about it, is being built incorrectly). The problem shouldn't affect runtime for this processing, but it does mean that I'll have to figure out how to deal with another damn problem that's always plagued me. Damn.
The real fun is finding out how much storage space this will all take with Storable, but that's next on the agenda.
That and figuring out how to pick words to start with. Once I've got that, it's just the painful process of letting it run.
