Lately, I have been working on the Oracle SuperCluster platform. After having worked with Linux for the past many years, it was quite refreshing to get back to an OS that so many of us have worked on. As part of our local zone layout, we have a requirement to allocate different amount of M7 CPU Threads per zone. Upon researching the best way to do this, I found varying information, so I thought that I would go ahead and blog about the way that worked best for this situation.
In this case, CPU Thread control was set-up using resource pools. Solaris Resource Pools are described here:
By default, the resource pool does not restrict access or control scheduling. By modifying the resource pool and allocating specific threads to specific zones, you thereby allocate threads to the local zones.
First, lets display the pool layout. Since we only need to look at allocating threads (the command actually outputs a ton of data), I will limit the output to only what is relevant.
Find the pool configurations you want to effect. Pset pertains directly to cpu threads so that is what we will look for:
# poolcfg -dc info | egrep 'pset |pset.size|pset.min|pset.max' pset pset_[host name]_id_25289 uint pset.min 32 uint pset.max 32 uint pset.size 32 pset pset_[host name]_id_25223 uint pset.min 64 uint pset.max 64 uint pset.size 64 pset pset_[host name]_id_25287 uint pset.min 64 uint pset.max 64 uint pset.size 64 pset pset_[host name]_id_25224 uint pset.min 32 uint pset.max 32 uint pset.size 32 pset pset_default uint pset.min 1 uint pset.max 65536 uint pset.size 64
In this case we can see that out of the 256 CPU threads available to this Global Domain, 32 have been allocated to the first local domain, 64 each to the next 2 and then 32 to the last, leaving 64 in the default pool or available to the global domain.
If you would like to see the file which also details the complete rules of the resource pool, you can look here:
To start with any modifications, it is best to ensure that the latest configuration is saved. To do so you can run this command from the global domain:
# pooladm -s
Once this has been done, you can proceed with the reallocation. In this example, I will modify one pool by taking CPU Threads from the default pool.
Using “-d” operates directly on the kernel state, so use this with caution. On a running system, I would reallocate in small chunks. That will give the operating system time to adapt to the different CPU configuration. In this example we will add 8 threads to a local zone which already had 32 Threads:
# poolcfg -dc 'modify pset pset_[host name]_id_25289 ( uint pset.min = 40 ; uint pset.max = 40)'
At this point the change has been made to the configuration file only (/etc/pooladm.conf), not actually to the system. To make the change to the system, save the configuration and commit to the system:
# pooladm -s # pooladm -c
Once this change is done, we can inspect the configuration by running the same command shown above. Notice the changes below:
# poolcfg -dc info | egrep 'pset |pset.size|pset.min|pset.max' pset pset_[host name]_id_25289 uint pset.min 40 uint pset.max 40 uint pset.size 40 pset pset_[host name]_id_25223 uint pset.min 64 uint pset.max 64 uint pset.size 64 pset pset_[host name]_id_25287 uint pset.min 64 uint pset.max 64 uint pset.size 64 pset pset_[host name]_id_25224 uint pset.min 32 uint pset.max 32 uint pset.size 32 pset pset_default uint pset.min 1 uint pset.max 65536 uint pset.size 56
If you need to transfer cpu from one local zone to another, you can do so by executing the following command:
poolcfg -dc 'transfer 8 from pset pset_default to pset_[host name]_id_25289'
Or if you want to assign a specific CPU Thread:
poolcfg -dc 'transfer to pset pset_[host name]_id_25289 ( cpu 5)'
The rest of the steps remain the same. In the next post I will show you how to verify the additional CPU in each local zone.