I have so many (hundreds of thousands) files and directories within this one specific directory that my "rm -rf" command to delete them has been taking forever.
I did this via the SSH, my question is: if my SSH connection times out before rm -rf finishes, will it continue to delete all of those directories?
This has happened to me on several occasions. Not a timeout per se, but the connection drops or something like that. There are ways to make the the rm process nearly immortal. But I simply reconnect and restart the command. The files that were deleted do not come back... they stay gone. So you don't lose any progress.
If this task is performed regularly, you should consider using a dedicated filesystem. When the time comes to wipe it, instead of rm use mkfs (or newfs, or whatever tool your system uses to create a filesystem).
Agreed, on some systems I actually do mkfs on boot for temporary cache partitions. Not just to clean it out, but to make it more robust -- the oft-changed SQUID cache is the most likely partition to go unfixably south if someone pulls the power for whatever reason.