I'm attempting to write a daemon that will start, stop, and monitor processes across a network of servers, meaning that a daemon would start on each server, attempt to connect to siblings at regular intervals (if there are unconnected siblings), and start services as remote dependencies are resolved.
This has been working fairly well thus far... I've utilized OpenSSL's PKI infrastructure to authenticate via public/private key pairs signed by a trusted CA (so only authorized clients can start/stop/monitor remote processes), and I've been able to track the PID of processes even after they fork (utilizing ptrace, I think similarly to the way the 'upstart' project works, though I haven't really had the chance to look at their code yet). What I'm running into a roadblock on, is process termination.
Basically, to follow a daemon's fork, my daemon will fork and exec the service daemon in question with a PTRACE_TRACEME, wait for a TRAP signal, and set the PTRACE_O_TRACEFORK | PTRACE_O_TRACEVFORK | PTRACE_O_TRACECLONE flags. It would then wait for another trap and check the cause by checking if:
((status >> 16) & 0xffff) == PTRACE_EVENT_FORK
((status >> 16) & 0xffff) == PTRACE_EVENT_VFORK
((statux >> 16) & 0xffff) == PTRACE_EVENT_CLONE
, and get the PID from the child with a PTRACE_GETEVENTMSG in those cases. This has been working beautifully. Once I get the final PID of the daemon in question, I detach from it with a PTRACE_DETACH and let it run unhindered. All good and well... but...
My daemon selects on a signal socket (via the signalfd function), listening for the SIGCHLD signal (which is supposed to be sent whenever a child terminates). This gets triggered, and I go into a series of waitpid(grandchild, &status, WNOHANG) calls on each monitored process to determine which one just terminated... but I get a "No child processes" thrown at me whenever I wait on a grandchild. I assume this worked before because I was using ptrace, and ptrace was attached... once I detached, the service daemon's original process became the parent, which then died, and then I am guessing the init process became the parent... meaning I probably can't use this approach at all because init becomes the parent.
I guess the bottom line... can I monitor a given grandchild process without becoming the init process (meaning I would need to write a replacement for init) ? I simply want to be able to detect when a grandchild terminates, so I can propogate that state across to sibling daemons across the network so they can react accordingly.
If I truly can't accomplish this without writing a replacement for init, is there some documentation somewhere on what exactly init needs to handle so I can write a proper replacement? I understand this is what the upstart project is doing, I just wish it had the capability to handle remote services.
EDIT: I've found this from Ubuntu about replacing init... they suggest starting from the source code for SysV init. I don't think I can do that... I'm pretty set on making my code BSD licensed, and IIRC, SysV is either GPL or some other form of incompatible license (please correct me if I'm wrong). I'd like to make sure this is as portable as possible, and I realize I may have to go about an alternative implementation of ptrace to follow forks if I want to be portable.