A simple way to find out is try out on some other shell script. Write a shell script that does nothing but sleep for a few seconds. (sleep 10). While it is being excuted in a terminal, from another terminal, delete/rename it.
Don't think you can count on that. Whether or not the shell keeps a local copy in memory is shell-specific, I've caused long-running BASH scripts to bomb out with syntax errors by modifying them while in execution. Strangely, deleting it may be safer than that -- if the shell already has a handle to the file, that handle will remain valid! The file will only truly be freed from disk once it's closed for the last time.
If you need to derive one script from another, make a copy, don't rip the floor out from under something else.