so server 1 is your global zone and server 2 is your non-global zone? are you inheriting all default filesystems from the global zone? my guess here is you are trying to write to a read-only fs. by default inherit-pkg-dir use the following filesystems as read-only loopback fs:
/lib
/platform
/sbin
/usr
i also saw something like this with oracle on a zone. the option was to create a whole root zone.
Sorry for the confusion with terminology, but, I am running one non-global zone with JBOSS App Server running as 2 separate instances (server 1 and server2) on the same Solaris server.
There are other zones on the server but for this application runs on one zone (accessing a database on another zone).
My point is that the application works correctly in a NON-ZONED Solaris 10 server with 2GB, but in a zoned environment I am getting the failure as stated (truss ENOMEM).
Are there any specific commands/settings I should be looking for at the Solaris level?
Either your zone has its swap size capped and you need to increase that size or you have just not enough swap. You can increase its size easily by adding a swap file.
As jlliagre mentioned, it is most likely a memory cap or just a memory issue. What's the physical memory/swap on this box and what's the output of:
#zonecfg -z <zoneName> info
If the info looks good (no cap on phys/swap) then check the memory usage from the global while you're starting up server2 to see if it's hitting the ceiling. Also try to just start server2 without having server1. Did you compare the max heap sizes of both servers?
Thanks for the 'simple' advise. Swap was indeed the problem, we upped it to twice the RAM size and I can now run 3 JBoss instances on the same Solaris 10 server.