Altho by default, solaris will create zvol under rpool for swap, one can use regular disk for it, a file or a zvol (3 options) from anywhere available.
Since your requirement is to have 3x1 TB of swap (??), i would go with disk backed devices.
Create a slice a on each disk, starting from 1st sector (not zero).
Add a swap device to configuration using swap -a <yourdisk><slice_number>
Modify the /etc/vfstab to have it after reboot.
Now you will have both ZVOL and new added disk listed under swap -l .
Remove the zvol via swap -d , the entry from /etc/vfstab , and using zfs command destroy the zvol.
The last operation might fail if swap is used (or has been used in past), then reboot will be required to remove it.
Question for you is, 3 TB of swap seems unreasonably high, why do you have requirement for such large swap space ?
Also, using fibre luns as swap could lead to terrible things to your infrastructure and storage, if you start using all that space...
How much memory do you have on that box ?
System has 2 TB physical memory and several zones are being build on it. The 3 TB swap comes from a recommendation to have 1.5 times RAM. This gave relief to our system which experienced severe performance issues.
This is what I did in my undertanding:
zpool create -m none extra c0t6d0 c0t6d1 c0t6d2
zfs create -p -V 2900G extra/swap2
zfs set primarycache=metadata extra/swap2
swap -a /dev/zvol/dsk/extra/swap2
swap -l
I would say that if you have a system with 2Tb of memory and have experienced performance issues, it is much more likely to be an application or configuration issue. A 3Tb swap area is inordinately large and as ZFS will use large portions of local memory for paging and swapping, moving significant ammounts of that to disk will impact performance of the SAN in a significant way.
I think that there is a need for some serious analysis work.
Huh, please point me where have you read that you need 1.5 X swap space, with 2 TB ram ?
I see no reason to have more then 1/4, which is more then enough.
Were they performance issues (like thrashing) or reliability issues (like coredumps)?
In general, increasing the swap area size is not expected to significantly improve performance (although in your case, the swap area is very undersized), but it can prevent programs to crash or to fail to start.
According to the statistics you wrote on the first post, there is 1.7T of virtual memory on your system with a tiny 4G being backed by your existing swap area.
The system is using 240 G of this virtual memory, two third of which being only reserved. You need at least 150 G of swap not to waste RAM (reserved size).
The actual swap size requirement depends on what precisely is running on your machine and each application way to use memory, but also about your dumpadm configuration. In case of a kernel panic, you might want to save the crashdump for further analysis and then a large swap area, like between 1/4 to 3/4 of the RAM size would make sense.
In any case, as long as the file systems also have enough free space, too large a swap doesn't hurt performance. There is a common misconception that a large swap will induce pagination. It doesn't make sense. What triggers pagination is RAM shortage, not swap availability.