I discovered yesterday that Module::Build supports recursive testing, meaning you can have subdirectories in your t/ directory.
So you can divide your .t files into subfolders following your namespace or whatever.
You simply have to out the following line in your: Build.PL
recursive_test_files => 1,