To begin with let me explain my system and then I will come to the problem.
System:
My program forks 2 child processes. These child processes further start 2 user level threads (pthreads) and 2 kernel level threads (kthread). All these threads issue various system calls. I am using Ubuntu with Linux kernel 2.6.39-rc7
Problem:
I want the following informtion about my system:
Scheduling information of the process and/or threads. ie. when was a particular process and/or thread scheduled.
Thread state information. ie. when and which thread is waiting for an I/O to get over. And is there a way, I can measure how much time does I/O takes?
Context switch information. ie. which system call caused the context switch and from which thread does this system call was called from?
Further, in case if I want to make changes to the linux scheduler and the context switch functionalities, then which files should I aim for?
I would really appreciate if you can answer all or part of these problems.
A context switch is not necessarily triggered by a system call. If that were the case, it would be trivial to write an infinite loop which would never release the cpu.
You probably want to begin with kern/sched.c, where you'll find scheduling and context switch code.
On an unrelated note, in the future, before starting a thread, choose the most appropriate forum and only post in that forum. Posting duplicates to multiple forums is frowned upon.
You can't come and go between user and kernel space.
These spaces are strictly separated.
The only form of communication from user space to kernelspace is with system calls.
User level threads (pthreads) create processes that run on userspace. They don't know what happens in kernel.
So to (1) no you can't know when or who scheduled the thread. If you mean when was created, then you have to provide this information on yourself on thread creation. man 7 pthreads shows shat info are stored and it basically ids, signals etc.
Also (2) you can't know if the thread is running or stopped (but you can know if it finished with nonblocking pthread_tryjoin_np()). However kernel does know, and publishes info on /proc fs (the same info you access with ps).
Finally, (3) you can't have context switching information from userspace (pthreads). Context switching is a kernel job. It is completely transparent to the userland. Of course, you can anytime give up processor from userspace with sched_yield() or from kernelspace by calling directly the scheduler with schedule() [i think].
As for scheduler, it resides at /usr/src/linux/kernel/sched.c
But it's a huge -- it's about 10k lines, you must really know what you are doing. And certainly with proper documentation.
ps: you can crash your machine very easily. There is no protection in kernel space. Try to find a program like vmware.
I am interested in doing the batch processing of system calls. My application has around 100 user level threads and around 64 kernel level threads. Whenever a user level thread requests a system call, instead of performing a system call I take a note of that system call and then I make that user thread sleep. Only when there are a considerable number of system calls waiting for me, i do a context switch and then let the kernel threads execute all those system calls. That ways, I am able to batch perform system calls and reduce the number of context switches. My application is working perfectly and I can see an improvement in performance.
However, now I actually want to trace through the application to see, whether it is doing exactly what I am expecting it to do. ie, when the user threads are executing only the user threads are executing and there is no switching to kernel threads and vice-versa. I also want to check when the system calls are being executed.
In reference to the problem that i posted, if I can trace through that dummy application then I can also trace through my real application.
I hope this helps and now you are in a better situation to provide more insightful information.
Thanks
---------- Post updated at 01:49 PM ---------- Previous update was at 01:36 PM ----------
Just to add.. Yes am working on VM, so crashing is not an issue
I'm being confused a bit.
How do you do "context switch"?
In userspace you can't.
In kernelspace you can, but to withhold the scheduler from scheduling kthreads would require to patch it, no? But you asked where the scheduler is in the kernel tree, so I assume you haven't.
So what do you context-switch?
Secondly: If a kthread starts to execute it cannot be scheduled-out. It will execute until finished or give up voluntary by calling schedule(). I don't know much about preemptive kernels (well, all are nowadays). I am sure that are very certain points of where a kernel preemption can occur -- after an interrupt for example. If you don't have an interrupt I can't imagine how the kthread could be unscheduled (by who??).
So what I think in a n-core pc:
pthread0...pthreadN sleep
kthread0...kthreadn-1 become runnable (as many as the cores)
kthreadx finishes
either a pthreadx or a kthreadx become runnable
Meanwhile, more kthreadx will finish.
Chances are as kthreadx finish, some pthreadx will became runnable.
So with time, all pthreads will become runnable and all kthreads will terminate.
But I am not 100% sure if everything works as I assume. I remember when I was programming a driver for the parallel port, upon entry to the interrupt service routine (triggered by a hardware interrupt) I did a CLI so no one could interrupt me. I did my job the fastest I could, then executed an STI and a schedule() to give up cpu. Preemptible kernel of not, there was no difference.
I don't seem to have the code to check it however, it's so many years that have passed.
Gotta go to bed() now because I'm cast to (void *) as I am sleepless for 2 days. Please be more specific (technically) and/or give or describe some code you have run.
Every system call is a context switch -- always. The very first thing that happens when you make a system call is the suspension of the calling thread or process. You can't avoid it. You can't prevent it.
Waiting for a mutex is a context switch, too. If you can grab it without waiting, that's usually not.
So threads sleep either way. You're actually putting more threads to sleep, for longer periods of time.
Without knowing what system calls you're actually doing when why, I'm only making wild guesses why this works better. But if I had to guess: Too many threads running simultaneously. Excessive context switching or cache synchronization eats into their time. Queueing some reduces the number of running threads, letting the rest get more work done with less switches or cache synchronization. As long as there's still more running threads than cores at all times, CPU isn't wasted.
If these system calls are disk-related, it could also be system cache effects.
