...and the dwarves keep tossing axes.
"Gee, I've got all this scaffolding here, STM should be easy to tack on!", says I to myself, proudly wielding my shiny new Thread::Sociable package as I dive headlong into the mire.
I'm now 2 solid weeks into some serious C hacking, and I've almost got the code done, though the original Sociable package is now in a bit of a shambles.
I've worked on the internals of massively parallel systems, reverse engineered protocol stack drivers from machine code, and written real-time operating systems in assembly. But this STM stuff is quite possibly the most mind-bending task I've ever encountered. (maybe because the "avoid premature optimization" rule can't really be applied, cuz Sociable is all about optimization.
With every line of code, I peer into the abyss of trying to quickly restart a transaction without inducing deadlock or livelock, and without eating the entire heap, all while desperately attempting to maintain the performance illusion - at least within a couple orders of magnitude - that its all just simple in-memory accesses.
Admittedly, scalars aren't too bad. If the sequence number is current, and, if needed, the thread can grab write ownership, the thread-private contents can be merged into the shared version toot-sweet. Even restart is pretty simple: just toss away the private version and start over.
STM for hashes is a bit pesky, but, since hashes carry significant overhead anyway, I'm not all that concerned about performance or memory. In reality, merging STM into hash support isn't too bad (tho a bit memory intensive): just a bit of list surgery.
But STM for arrays is a damnable nuisance. The mounds of code needed to manage all the possible transactional operations has become a Tower of Babel. Array access is supposed to be fast, dammit!
Yet, as I duct tape ever more code onto the pile, my "code at the traces" persona cringes at the thought of someone loading up a 10,000 element array and iterating over the whole thing inside a transaction...esp. when there are 10 or 20 other transactions trying to do the same thing to the same array at the same time.
Oh well. The code is nearly done, tho I've got a lot of repairs to get Sociable back to baseline functionality. Then the Aegean stable of testing/debugging STM begins...
One upside: glueing this underneath DBD::RAM should make for an interesting little multi-user main memory database.
