There is no predifined directory for custom software.
Mostly the installation destination is defined with the setup/installation.
If the mount points do not sound like they could house the application, you might want to use the find command, maybe with -mtime switch etc. or have check the process list with ps for the name of processes that belong to the application. If you are lucky, you might see the path to the program.
To begin with, its an addition to what zaxxon has explained,
Who installed it? if its root and if you have installed it using (smitty or installp), you can run lslpp to determine installed version.
And if its an application installed by development/application user then its a different story.
->I checked on the server using lslpp and find command and I am able to see the software version.
-> but once again, I have one more Question, if I dont know the third party software name, then how to differenciate between server inbulid software and third party software.
As mentioned above, your best bait is lslpp , if its a third party software not installed by root, then it wont show up. If you don't have the name and you like proof reading then run lslpp -l to read all the installed software.
But on the other hand, a smart Admin with install base OS with minimum packages (that are really required).
lslpp -f filename will show you the files that belong to the fileset and give you the paths and directories.
If you want to do a find search try
find / -name *filename* -print and troll through the output.