First (but less important), to ease up on memory usage, convert the comment tree traversal stuff to be iterative, not recursive. This isn't very difficult - rather than making a recursive function call, just maintain a stack of the current location in the comment tree and push/pop by yourself.
Second (and this is where the important stuff happens), have two limits (configured for the server, but have it possible for the user to specify lower limits if they like): depth and count. Depth simply restricts how deep the comment traversal can go, and count simply restricts how many total comments can be displayed.
When a depth limit (which should probably be around 20) is hit, instead of displaying the child comments, display a message like "Continued..." which is a link to the last comment which was displayed.
When the total comment limit (which should probably be around 250) is hit, continue traversing comments until you get back up to the top display level, and display a message like "Continued..." which is a link to displaying the comments, starting traversal at the first toplevel comment to not be displayed.
Implementing these changes shouldn't be too difficult (in fact, I might even download the codebase to try at it), but it seems like a much better solution than Slash's "overload mode." It wouldn't break the continuity of discussions, it wouldn't overload Scoop due to having to touch a large amount of the comment database or due to extreme recursion, and it'd present a friendly, consistent interface which prevents browsers from getting really bad.
