tag:blogger.com,1999:blog-35472124602418424662024-02-08T19:10:56.995-07:00VIOS for IBM i BlogUnknownnoreply@blogger.comBlogger6125tag:blogger.com,1999:blog-3547212460241842466.post-87318826870823568442023-06-13T11:51:00.000-07:002023-06-13T11:51:11.686-07:00Proactive vNIC Changes for VIOS Maintenance<p>In January 2023, I published an article and shared Python
code that allows monitoring of your systems to verify your vNIC backing devices
are configured for the best possible redundancy and warn you of any situations
that should be resolved.</p><p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal">I just published a new script that extends on that idea to
allow you to proactively change backing devices so you can free a VIOS for
maintenance (upgrades, etc.)<span style="mso-spacerun: yes;"> </span>While you
can certainly use the HMC Web based GUI to view and change individual vNIC
backing devices, it can be a time-consuming process if you have a lot of devices
to change.<span style="mso-spacerun: yes;"> </span>Of course, failover is
automatic if properly configured, so you could just shut down the VIOS and let
the failover handle the switching, but many people prefer a more planned and
controlled approach.<o:p></o:p></p>
<p class="MsoNormal">This script has two primary functions:</p><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="text-indent: -0.25in;">Change all vNIC devices for a specified Power
server so any active backing devices associated with a specified VIOS are
changed to the highest priority (lowest numbered) operational alternative
backing device that is NOT served by the specified VIOS.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">In other words, move all vNICs off a specified
VIOS so that VIOS can be maintained.</span></li><li>Change all vNIC devices for a specified Power
server to set the auto priority failover flag to either 1 or 0.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">This is intended to make it easy to undo the
previous usage.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">When you force a
specific backing device, auto priority failover is automatically set to 0 to
prevent the system from switching right back to the original backing
device.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Setting it back to 1 (on) after
the maintenance is complete will put all the backing devices back to the preferred
interfaces based on priority.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">I usually
recommend setting auto priority failover to 0 (off) during normal operations to
prevent flapping between interfaces in the case of intermittent failure, and
this script can be used to do that as well.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">If you choose to do that, I strongly recommend regularly monitoring for
non-operational interfaces using my previously published monitoring script </span><a href="https://blog.vios4i.com/2023/01/monitoring-vnic-on-power.html" style="text-indent: -0.25in;">https://blog.vios4i.com/2023/01/monitoring-vnic-on-power.html</a><span style="text-indent: -0.25in;">
or another monitoring tool or process.</span></li></ul><p></p><p class="MsoListParagraphCxSpLast" style="mso-list: l4 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoNormal">If you need more background on vNIC, please see my previous
article: <a name="_Hlk137544999"></a><a href="https://blog.vios4i.com/2022/11/sriov-and-vnic.html"><span style="mso-bookmark: _Hlk137544999;">Introduction to SR-IOV and vNIC for IBM i</span></a><span style="mso-bookmark: _Hlk137544999;"></span>. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>Getting the vnic-move.py script<o:p></o:p></h2>
<p class="MsoNormal">You can find the open source script vnic-move.py in the
public repository at: <a href="https://github.com/IBM/blog-vios4i">https://github.com/IBM/blog-vios4i</a><o:p></o:p></p>
<p class="MsoNormal">Download it directly with: <a href="https://github.com/IBM/blog-vios4i/raw/main/src/vnic-move.py">https://github.com/IBM/blog-vios4i/raw/main/src/vnic-move.py</a><o:p></o:p></p>
<p class="MsoNormal">This is a free open-source script released under Eclipse
Public License v2.0.<span style="mso-spacerun: yes;"> </span>Bug fixes and
improvements will be checked into the public Git repository when tested.<span style="mso-spacerun: yes;"> </span>If you want to monitor for changes, I suggest
creating a github account and watching the project as it is unlikely I’m going
to write an article about each change.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>Setting up the vnic-move.py script<o:p></o:p></h2>
<p class="MsoNormal">Unlike the vnic-check.py monitoring script, this script is
intended to be used interactively by a system administrator only when preparing
to perform maintenance on a VIOS.<span style="mso-spacerun: yes;"> </span>That
means, while it is possible to run this on an IBM i using the PASE environment,
it is much more likely that this will be run from an administrator’s
workstation.<span style="mso-spacerun: yes;"> </span>Given the current sad state
of world where Windows is the most widely used desktop operating system, that
poor System Administrator will probably be forced to use Windows rather than
something better (*cough* Linux *cough*).<o:p></o:p></p>
<p class="MsoNormal">If you are going to run this on AIX or Linux, install
Python3, then create keys and run the script as shown below.<o:p></o:p></p>
<p class="MsoNormal">If you are running Windows, you have a few options to run
Python3 including (easiest to hardest) as a native Windows Executable, in a
container using a container manager like Docker, with the Windows Subsystem for
Linux (WSL), or as a Linux Virtual machine.<span style="mso-spacerun: yes;">
</span><o:p></o:p></p>
<p class="MsoNormal">This script runs commands on the HMC using the ssh command,
so you will also need to verify you have that command in the environment where
you will run it (but don’t despair if you don’t).<span style="mso-spacerun: yes;"> </span>The good news is that even Windows 10/11
generally has the ssh command.<span style="mso-spacerun: yes;"> </span>To find
out if this is true in your case, just open a command line and run “ssh”.<span style="mso-spacerun: yes;"> </span>If you get a usage message, you’ve got it.<span style="mso-spacerun: yes;"> </span>If it’s not there, Windows
Settings->Apps->Optional Features will usually let you install “OpenSSH
Client” unless your organization has other ideas.<o:p></o:p></p>
<h2>Setting up the ssh keys and agent<o:p></o:p></h2>
<p class="MsoNormal"><o:p> </o:p>This script runs remote HMC commands via a batch mode SSH
command, so you will need to configure an SSH key to avoid a password prompt. This key can either have an empty passcode or
a secure passcode using an ssh-agent. To
be clear, I would never recommend using an empty passcode ssh key for a user
account that can make changes to your environment, so the choice I recommend is
using an ssh-agent to manage access with a passcode.</p><p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal">In general, the process you will need to use is:</p><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="text-indent: -0.25in;">Create an account on the HMC that you will use
for this script.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">You can skip this step
if you already have separate accounts for each system administrator, or if you
are okay will running the command with the default hscroot account.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Please note that the HMC account will need
permissions to run the lshwres command to retrieve the vNIC information, and to
run the chhwres command if you want to actually switch the backing devices.</span></li><li>Generate a public key/private key on your
workstation (or where you want to run the script).<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Usually, this is done with the ssh-keygen
command, and usually is just a matter of running the commands and responding to
the prompts. Mostly with the defaults.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">If
you leave the passcode blank (not recommended), you will not need to do any of
the ssh-agent stuff below.</span></li><li>If you selected defaults, the ssh-keygen command
will have created an id_rsa.pub file, and it will have showed you where it
created it.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">You will need to add this
public key to the authorized keys of the HMC account that you want it to
use.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">The correct way to do that on the
HMC is from the command line with mkauthkeys.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">The format is: mkauthkeys -a “[contents of public key]”.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">The easiest way to do this is probably to
open the public key file with notepad and copy/paste it to the command.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">If you do the copy/paste thing, please note
that the public key is one long string with no embedded lines, so pay attention
to wrapping in your text editor.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">If you
see “>” continuation lines when running the command on the HMC, you probably
included a line break that shouldn’t be there.</span></li><li>Test the public key access from the workstation with
the command: “ssh [hmcaddress]”<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">The
first time you run this it will prompt you if you want to trust that host.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">You will need to respond yes so that the host
key is added to your known_hosts file.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">It should prompt for your passcode, and when that is provided, it will
give you access to HMC command line.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">The
exit command will end the ssh session. </span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Repeat
a second time to verify that it skips the host verification prompt.</span></li><li>Setup your ssh-agent</li><ul><li>If running Unix (Container, WSL, or VM), you’ll
just run: “ssh-agent [shell]” where shell is usually bash.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">This will give you a shell that is a child of
the ssh-agent, so you can proceed with the add keys option below.</span></li><li>If running native Windows, there are a few more
steps.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">First you’ll need to go into your
services app -- “services.msc” will get you there from the search line.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Find the service named “OpenSSH
Authentication Agent” and make sure it is not Disabled.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">“Automatic (Delayed Start)” is a good choice
as it will only open when needed.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">You
only need to enable the service once.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">After that, just run “ssh-agent” from the command line to start it each
time you need to use it.</span></li></ul><li>Authenticate your keys to the agent.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">No matter what method you use, this is done
with the command “ssh-add [path to id_rsa file]”</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">For Windows users, you’ll probably need to
give the whole path to the key file, for Unix users, you can usually use:
~/.ssh/id_rsa.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">When it finds the file,
it will prompt for the passcode you set.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">When you enter the passcode, it will store the unlocked key in the agent
service memory.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Be aware that when using
an agent, realistically any process on the computer where it is running can
access the unlocked keys.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Save your
risky web activities (you know what I mean) for times that you have not
unlocked the keys.</span></li><li><span style="text-indent: -0.25in;">While the agent is running and the key is
unlocked, the ssh command in the script will access the unlocked key from the
agent and skip the prompt for a passphrase.</span><span style="text-indent: -0.25in;">
</span><span style="text-indent: -0.25in;">This allows the script to run all the commands in batch mode as it needs
to.</span></li><li>When you are done with the unlocked keys, you
can start over by running “ssh-agent -D” to remove all unlocked keys.<span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">This is especially important on a shared
workstation, or one that you never log off.</span></li></ul><p class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoNormal">If you have any problems getting the ssh keys and
authentication setup, google is a great resource.<span style="mso-spacerun: yes;"> </span>Secure shell has been around a long time, so
many people have tackled the process of setting up keys and using an agent, and
some have written good tutorials on how to do it.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>Using the vnic-move.py script<o:p></o:p></h2>
<p class="MsoNormal">Now that the ssh and authentication setup part is out of the
way, here’s some examples of how to use the script:</p><p class="MsoNormal"><b><span style="text-indent: -0.25in;">python3 vnic-move.py --hmc myhmcuser@myhmc --vios=vios1
--system myp10system –verify</span> </b></p>
<p class="MsoNormal">We would use this if we are planning on taking vios1 down
for maintenance.<span style="mso-spacerun: yes;"> </span>This one will check all
the vNICs for system mkp10system managed by myhmc and generate the commands to
change the backing devices for any vnic that currently have a backing device
served by vios1.<span style="mso-spacerun: yes;"> </span>The –verify option
makes it check the configuration and print the commands without executing
them.<span style="mso-spacerun: yes;"> </span>You can then manually run the
commands, or run the next one to run them all automatically.</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py --hmc myhmcuser@myhmc --vios=vios1
--system myp10system</b></span></p><p class="MsoNormal">This one will do what the
previous one did, plus it will run the commands to make the vnic device changes
on the HMC.</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py --hmc myhmcuser@myhmc --vios=vios2
--system myp10system</b></span></p><p class="MsoNormal">Suppose now we have run the previous command above and
shutdown vios1, here is the command to do vios2., but what if vios1 is not
finished starting up when we tell it to do vios2? If there are three valid backing devices via
three or more vios, it will happily switch to the alternate vios and continue. If not, it is going to print an error telling
you there is no operational alternate backing device and stop without running
any commands. This error could also display
if you happen to have one test system that only has one backing device. If you have reviewed the error messages and
know that you don’t care if they lose connectivity, perhaps you know that vnic
is not critical if it loses connectivity or it is powered off, you can skip all errors with the --force
option:</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py --hmc myhmcuser@myhmc --vios=vios2
--system myp10system --force</b></span></p><p class="MsoNormal">If you force changes and it breaks something, that’s on
you. To be clear, you really should look
at all the commands generated and make sure you are comfortable with running
them in any case, because any code can have errors, and there are no warranties
or support contracts for this free open-source script.</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py --hmc myhmcuser@myhmc --system
myp10system –autofailover=1</b></span></p><p class="MsoNormal">Suppose now you’ve finished all of your maintenance and all
VIOS are back online, so now you want to reset auto-priority-failover back on
so everything is running where it should.
The above command will do that.</p><p class="MsoNormal"><br /></p><p class="MsoNormal">All of those are great if you are not working in a highly
restrictive security environment, but maybe your employer won’t allow you to
install Python on the workstations with ssh access to the HMC, or they have a
blanket policy against using ssh keys (I’m not going to judge). There is still an option to use this script
to make your life easier. Starting with
the first example, pending maintenance on vios1:</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py –offline --vios=vios1</b></span></p><p class="MsoNormal">That will print the command you need to run on hmc myhmc:</p><p class="MsoNormal"><i>Collect data from HMC with the following command and store
in a file:</i></p><p class="MsoNormal"><i><o:p></o:p></i></p>
<p class="MsoNormal"><i>lshwres -m myp10system -r virtualio --rsubtype vnic --header
-F lpar_name%lpar_id%slot_num%auto_priority_failover%backing_devices%backing_device_states</i><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">You can copy/paste or otherwise transfer the command to the
system that can run ssh and then copy/paste the output of the command to a file
on the local workstation where you are running the script.<o:p></o:p></p>
<p class="MsoNormal">Now you can process that file with the following:</p><p class="MsoNormal"><span style="text-indent: -0.25in;"><b>python3 vnic-move.py –file=/path/to/file --vios=vios1
--system myp10system</b></span></p><p class="MsoListParagraph" style="mso-list: l2 level1 lfo5; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoNormal">That will check the input and print the commands needed to
change the vNIC backing devices.<span style="mso-spacerun: yes;">
</span>Copy/paste or otherwise transfer and run those commands and you will be done
with that step.<o:p></o:p></p>
<p class="MsoNormal">If you use offline mode like this, make sure you don’t allow
too much time between collecting the command output and processing it or you
might generate commands that are no longer correct for the CURRENT state of the
vNIC devices.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>Need help?<o:p></o:p></h2>
<p class="MsoNormal">If you need help implementing best practices for your vNICs,
the IBM i Technology Expert Labs team (formerly known as Lab Services) is
available to help with implementation planning, execution, and knowledge
transfer.<span style="mso-spacerun: yes;"> </span>See <a href="https://www.ibm.com/services/infrastructure">https://www.ibm.com/services/infrastructure</a>
for contact information or speak to your IBM Sales Representative or Business
Partner.<span style="mso-spacerun: yes;"> </span>If you are planning a new
hardware purchase, you can include implementation services by the Technology Expert
Labs team in your purchase.<o:p></o:p></p>
<h2>Disclaimer<o:p></o:p></h2>
<p class="MsoNormal">I am an employee of IBM on the IBM i Technology Expert Labs team
(formerly known as Lab Services).<span style="mso-spacerun: yes;"> </span>The opinions
in this post are my own and don't necessarily represent IBM's positions,
strategies, or opinions.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>References<o:p></o:p></h2>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Previous Blog post on vNIC and SR-IOV<o:p></o:p></p>
<p class="MsoNormal"><a href="https://blog.vios4i.com/2022/11/sriov-and-vnic.html">https://blog.vios4i.com/2022/11/sriov-and-vnic.html</a></p>
<p class="MsoNormal"><br /></p><p class="MsoNormal">Previous Blog post with a vNIC monitoring script<o:p></o:p></p>
<p class="MsoNormal"><a href="https://blog.vios4i.com/2023/01/monitoring-vnic-on-power.html">https://blog.vios4i.com/2023/01/monitoring-vnic-on-power.html</a><o:p></o:p></p>
<p class="MsoNormal"><br /></p><p class="MsoNormal">Github public repository for this Blog<o:p></o:p></p>
<p class="MsoNormal"><a href="https://github.com/IBM/blog-vios4i">https://github.com/IBM/blog-vios4i</a><span class="MsoHyperlink"><o:p></o:p></span></p>
<p class="MsoNormal"><span class="MsoHyperlink"><o:p><span style="text-decoration: none;"> </span></o:p></span></p>
<p class="MsoNormal">Microsoft Article on using Public keys with Windows<o:p></o:p></p>
<p class="MsoNormal"><a href="https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement">https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM Support page on setting up SSH keys for the HMC<o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/setting-ssh-run-commands-hardware-management-console-without-being-prompted-password">https://www.ibm.com/support/pages/setting-ssh-run-commands-hardware-management-console-without-being-prompted-password</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3547212460241842466.post-72728655546843719402023-03-16T08:37:00.002-07:002023-03-19T10:29:40.390-07:00New tool to work with WWNN/WWPN Really quick post here. <div><br /></div><div>To be honest, I'm doing this as much for my own use and to make life a bit easier for my customers as we work through setups. </div><div><br /></div><div>For a while now, I've been using a little local HTML/Javascript tool I made to add or remove colons and switch upper/lower case on WWPNs. Anyone that has done much work with multiple platforms where some require lowercase with colons and others want uppercase without colons knows the pain. </div><div><br /></div><div>There are plenty of these already available on the web, but in my <a href="https://lmgtfy.app/?q=wwpn+convert+tool">quick search</a>, I didn't find one to do exactly what I wanted, so I wrote this one to use offline.
For example, I frequently copy a storage adapter WWPN from IBM i DSPHDWRSC or STRSST to Brocade switches or DS8000/FlashSystems storage platforms to create zoning or host groups. The WWPN you get from IBM i is all uppercase with no colons, while the Brocade switches want to get lowercase with colons. </div><div><br /></div><div>This tool will let you easily convert a WWN or list of WWNs to/from any combination of upper/lowercase or colons/no-colons.
Bookmark the URL <a href="https://blog.vios4i.com/p/wwn-tool.html">https://blog.vios4i.com/p/wwn-tool.html</a> and use it whenever you need, or just come to the <a href="https:/blog.vios4i.com">https:/blog.vios4i.com</a> home page and use the permanent link on the right side of the screen, then stick around and read and comment on my latest blog post. </div><div><br /></div><div>If you'd also like to have a tool to use offline, this one does all of its work with HTML and embedded javascript. I've made it available on my github <span style="font-family: inherit;">repository</span>: <span style="font-family: inherit;"><span style="background-color: white;">You can find the open source wwpncvt.html in the public repository at: <a href="https://github.com/IBM/blog-vios4i">https://github.com/IBM/blog-vios4i</a></span></span></div><div><span style="font-family: inherit;"><span style="background-color: white;"><br /></span></span></div><div><span style="font-family: inherit;"><span style="background-color: white;">Download directly with: <a href="https://github.com/IBM/blog-vios4i/raw/main/src/wwpncvt.html">https://github.com/IBM/blog-vios4i/raw/main/src/wwpncvt.html</a> Use Save As to get a local copy and run that without any internet connection.</span></span></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3547212460241842466.post-41960913982609196292023-01-14T11:20:00.000-07:002023-01-14T11:20:19.620-07:00Monitoring vNIC on Power<p> </p><p class="MsoNormal">As I discussed in my previous post <a href="https://blog.vios4i.com/2022/11/sriov-and-vnic.html">Introduction to
SR-IOV and vNIC for IBM i</a>, vNIC is a powerful feature built on top of
SR-IOV and VIOS that virtualizes Ethernet connections to your IBM i, AIX, or
Linux partitions with high performance and automatic failover to alternative
ports (backing devices).</p><p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal">One problem with creating a vNIC with multiple backing
devices in a failover configuration is you might not notice when it
automatically switches to a backup device because a higher priority device
fails.<span style="mso-spacerun: yes;"> </span>If you don’t notice, you could
end up running in a reduced redundancy situation because the problem that
caused the switch never gets resolved.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">The solution for that problem is monitoring.<span style="mso-spacerun: yes;"> </span>To that end, I’m publishing a Python script I
wrote to monitor your vNIC configuration for several conditions that could
indicate a problem.<o:p></o:p></p>
<h2>Best practices<o:p></o:p></h2>
<p class="MsoNormal">When creating a vNIC configuration, there are a few
practices that will help ensure you get the best possible redundancy to protect
your self from failures in various parts of your infrastructure.</p><p class="MsoNormal"></p><ul style="text-align: left;"><li><span style="text-indent: -0.25in;">Use multiple backing devices.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Each vNIC should have at least one backing
device to failover to in case the primary device encounters a problem.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">It is common to see configurations with 3 or
4 total backing devices.</span><span style="text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">As of this
writing, up to 16 are supported.</span></li><li>Spread backing devices across points of
failure.<span style="mso-spacerun: yes;"> </span>Separate them to different
VIOS, different adapters, and different network switches.</li><li>Spread active devices across all VIOS and
physical adapters to statically load balance the work.<span style="mso-spacerun: yes;"> </span>Don’t put all your active devices on one VIOS
and all your backup devices on the other or you’ll be looking a big
processor/memory spike when you take the primary VIOS down.<span style="mso-spacerun: yes;"> </span>Likewise, it makes little sense to squeeze
all your traffic through some of your adapters/ports and leave others
idle.<span style="mso-spacerun: yes;"> </span>Using all of your ports also makes
it possible to detect port failures that are related to switching and routing
that would otherwise go undetected until they are your only option when primary
ports fail.</li><li>Assign each backing device a unique failover
priority so the backup sequence is deterministic.<span style="mso-spacerun: yes;"> </span>For vNIC, the lowest priority number is the
highest priority, and it defaults to 50.<span style="mso-spacerun: yes;">
</span>Typically, I would assign 50 to the desired active device, 60 to the
first backup, 70 to the second backup, etc.<span style="mso-spacerun: yes;">
</span>You can use any numbers you wish, but keep them unique for a given vNIC,
and leave some space in the numbering to change it around if you need to.</li><li>Use the HMC GUI to activate backup ports if you
need to move traffic proactively.<span style="mso-spacerun: yes;"> </span>Select
a specific LPAR from the HMC screen, then select Virtual NICs from the left-hand
menu to display/edit the vNICs for a partition.<span style="mso-spacerun: yes;">
</span>To switch to a different backing device, select the desired device and
select Action-Make the backing device active.<span style="mso-spacerun: yes;">
</span>You will notice that when you do this, the “Auto Priority Failover” setting
will change to “Disabled”. That will prevent the vNIC from switching based on
priority unless the active port fails.</li></ul><p></p><p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><o:p></o:p></p>
<h2>Installing the Monitor code<o:p></o:p></h2>
<p class="MsoNormal">The vnic-check.py script is written in Python.<span style="mso-spacerun: yes;"> </span>It will run on any platform that has
Python3.6 or above, including the IBM i.<span style="mso-spacerun: yes;">
</span>As this is an IBM i centric blog, I include the instructions to install,
configure and run it on IBM I using the PASE environment, but it will run an
pretty much anywhere.<o:p></o:p></p>
<h3>Prerequisites<o:p></o:p></h3>
<p class="MsoNormal">You must have the PASE environment (5733-SC1 Base), and
OpenSSH installed (5733-SC1 Opt 1). <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h3>Installing Open Source packages and python3<o:p></o:p></h3>
<p class="MsoNormal">See <a href="https://www.ibm.com/support/pages/getting-started-open-source-package-management-ibm-i-acs">https://www.ibm.com/support/pages/getting-started-open-source-package-management-ibm-i-acs</a>
for details of this process.<o:p></o:p></p>
<p class="MsoNormal">Start the SSH server on the IBMi partition with the STRTCPSVR
*SSHD command if it is not already started.<o:p></o:p></p>
<p class="MsoNormal">Using IBM Access Client Solutions (ACS),<span style="mso-spacerun: yes;"> </span>Select a system for the LPAR where you wish
to install the monitoring tool.<span style="mso-spacerun: yes;"> </span>From the
menu, select Tools->Open Source Package Management<o:p></o:p></p>
<p class="MsoNormal">On the “Connect to SSH” window that is displayed, enter a
user ID and password with sufficient authority to install the open source
packages (see link above for details)<o:p></o:p></p>
<p class="MsoNormal">If your IBMi partition does not have the ability to connect
to the Internet, use the radio button under “Proxy Mode” to select “SSH
Tunneling” This will allow the packages to be downloaded via your workstation.<o:p></o:p></p>
<p class="MsoNormal">If you get a message box that the Open Source Environment is
not installed, click “Yes” to install it.<o:p></o:p></p>
<p class="MsoNormal">When the open-source environment install is complete, it
will display a window with the list of installed packages.<span style="mso-spacerun: yes;"> </span>If python3 is in that list, you are
done.<span style="mso-spacerun: yes;"> </span>If not, switch to the “available
packages” tab, click “python3” and click Install.<span style="mso-spacerun: yes;"> </span>If no available packages display, you may
need to close the open source management window and reopen it.<o:p></o:p></p>
<p class="MsoNormal">Verify python3 is installed by opening a QShell session (QSH
command) and running “/QOpensys/pkgs/bin/python3 -V”<span style="mso-spacerun: yes;"> </span>It should show a Python version number of 3.6
or higher.<span style="mso-spacerun: yes;"> </span>F3 to exit back to your
command line.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h3>Create an HMC Monitoring account<o:p></o:p></h3>
<p class="MsoNormal">This script runs query commands on the HMC using a private
key with no passphrase.<span style="mso-spacerun: yes;"> </span>Since it is a
<b>very </b>bad security idea to have that kind of access to your HMC hscroot account,
you’ll want to create an account that can only run monitoring commands.<span style="mso-spacerun: yes;"> </span>Seriously, DO NOT create private keys as
described here using hscroot or any other HMC account that can make
changes.<span style="mso-spacerun: yes;"> </span>If you’re not going to use a
restricted account, don’t use this script.<o:p></o:p></p>
<p class="MsoNormal">Connect to the HMC using a SSH client like Putty as user
hscroot (or another user with the authority to create user accounts).<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">Run the command:<o:p></o:p></p>
<p class="MsoNormal">mkhmcusr -i "name=monitor,taskrole=hmcviewer,description=For
restricted monitoring
scripts,pwage=99999,resourcerole=ALL:,authentication_type=local,remote_webui_access=0,remote_ssh_access=1,min_pwage=0,session_timeout=0,verify_timeout=15,idle_timeout=120,inactivity_expiration=0"<o:p></o:p></p>
<p class="MsoNormal">It will prompt for a password.<span style="mso-spacerun: yes;"> </span>Assign it something secure.<span style="mso-spacerun: yes;"> </span>This will create an account named “monitor”
that can only be used by the SSH interface.<span style="mso-spacerun: yes;">
</span>It will not be able to use the Web GUI, and it will be restricted in the
commands it can run.<o:p></o:p></p><p class="MsoNormal">Repeat this account creation on each HMC that will be monitored.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<h3>Create a key to access the monitor account<o:p></o:p></h3>
<p class="MsoNormal">You will be running the monitoring script from one of your
IBMi partitions, with a specific user id that will have an SSH key to access
the HMC using the monitor account you just created.<o:p></o:p></p>
<p class="MsoNormal">Pick the User-Id that will be running the command.<span style="mso-spacerun: yes;"> </span>I’m not going to go into detail on creating
this account since if you’re an IBMi administrator, you already know how to
create accounts and create job schedule entries that use a specific account.<span style="mso-spacerun: yes;"> </span>Of course, you can use an existing account
for this as well.<o:p></o:p></p>
<p class="MsoNormal">The account you choose will need to have a home directory
where you can create an ssh private key that you will authorize to connect to
the HMC monitor account.<o:p></o:p></p>
<p class="MsoNormal">Start QShell (QSH) from the account you will use and run the
following:<o:p></o:p></p>
<p class="MsoNormal"># on all of the following commands, replace 1.2.3.4 with the
IP address of the HMC you want to monitor. Repeat for each HMC if you are monitoring more than one.<o:p></o:p></p>
<p class="MsoNormal">mkdir -p $HOME # make sure there is a home directory<o:p></o:p></p>
<p class="MsoNormal">cd $HOME # change to the home directory<o:p></o:p></p>
<p class="MsoNormal">ssh-keygen<o:p></o:p></p>
<p class="MsoNormal"># press enter three times to accept the default file /home/MONITOR/.ssh/id_rsa
and use an empty passphrase<o:p></o:p></p>
<p class="MsoNormal">ssh monitor@1.2.3.4 mkauthkeys -a \"`cat
~/.ssh/id_rsa.pub`\"<o:p></o:p></p>
<p class="MsoNormal"># answer ‘yes’ to the authenticity prompt<o:p></o:p></p>
<p class="MsoNormal"># Enter the HMC Monitor account password when prompted for
Password:<o:p></o:p></p>
<p class="MsoNormal"># finally test the SSH key access with:<o:p></o:p></p>
<p class="MsoNormal">ssh monitor@1.2.3.4 lssyscfg -r sys -F name<o:p></o:p></p>
<p class="MsoNormal"># you should get a list of the system names managed by that
HMC without any password prompting<o:p></o:p></p>
<p class="MsoNormal">Use F3 to leave the QShell prompt.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Now you’ll need to download and edit the top of the
vnic-check.py file to set your parameters.<span style="mso-spacerun: yes;">
</span><o:p></o:p></p>
<p class="MsoNormal">You can find the open source script vnic-check.py in the
public repository at: <a href="https://github.com/IBM/blog-vios4i">https://github.com/IBM/blog-vios4i</a><o:p></o:p></p>
<p class="MsoNormal">Download directly with: <a href="https://github.com/IBM/blog-vios4i/raw/main/src/vnic-check.py">https://github.com/IBM/blog-vios4i/raw/main/src/vnic-check.py</a></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>smtphost</b>: Set to the name (or address) of a SMTP relay in
your organization where you can send mail.<span style="mso-spacerun: yes;">
</span>On the IBMi, if the current partition is running a mail server locally,
you can use 127.0.0.1 here.<span style="mso-spacerun: yes;"> </span>Set this to
None if you just want to get a printed result to the screen (or a spooled file
in batch).<span style="mso-spacerun: yes;"> </span>Using None is useful to
ensure the command is working properly before setting up email.<o:p></o:p></p>
<p class="MsoNormal">sender:<span style="mso-spacerun: yes;"> </span>If using
email, this needs to be a valid email address that can send mail in your
organization.<o:p></o:p></p>
<p class="MsoNormal"><b>toaddrs</b>:<span style="mso-spacerun: yes;"> </span>This is a
list of email addresses that should get messages when the check finds any
conditions that need fixing.<span style="mso-spacerun: yes;"> </span>You can use
a comma separated list of addresses between the brackets where each address is
enclosed in quotes.<o:p></o:p></p>
<p class="MsoNormal"><b>hmcs</b>: this should be a list of the SSH address of the
monitor account on your HMC in the format monitor@ipaddress.<span style="mso-spacerun: yes;"> </span>You can also use a DNS name instead of the ip
address if DNS is properly configured for your PASE environment (verify by using the host command in a PASE shell).<span style="mso-spacerun: yes;"> </span>The entire list should be surrounded by []
characters, and each hmc address should be surrounded by single quote
characters and separated by commas.<span style="mso-spacerun: yes;"> </span>It
is okay to only have one hmc in the list.<span style="mso-spacerun: yes;">
</span>You will need to do the same key setup described above on each HMC if you use more than one.<o:p></o:p></p>
<p class="MsoNormal"><b>minopercount</b>: this should be the lowest number of backing
devices that is acceptable in your environment.<span style="mso-spacerun: yes;">
</span>Any vNIC with less than this number of operational devices will be
reported as a problem.<o:p></o:p></p>
<p class="MsoNormal">When you have set your parameters, transfer the script to
the home directory of the user that will be running the command.<o:p></o:p></p>
<p class="MsoNormal">Finally, make sure it works by opening QShell (QSH command)
and running the script:<o:p></o:p></p>
<p class="MsoNormal">/QOpensys/pkgs/bin/python3<span style="mso-spacerun: yes;">
</span>vnic-check.py<o:p></o:p></p>
<p class="MsoNormal">If all goes well, you’ll get no email or output (indicating
all of the vNICs found are without problems), or a list of the problems
found.<span style="mso-spacerun: yes;"> </span>If you get no output and want to
make sure it is finding your vNICs,<span style="mso-spacerun: yes;">
</span>Change the minopercount variable to a high number (999) and rerun to
report all of your vNICs are lower than the desired count.<o:p></o:p></p>
<p class="MsoNormal">When you have verified all is well, reset the variables as
needed and schedule a job to run: <o:p></o:p></p>
<p class="MsoNormal">QSH CMD('/QOpensys/pkgs/bin/python3<span style="mso-spacerun: yes;"> </span>vnic-check.py')<o:p></o:p></p>
<p class="MsoNormal">as the selected user on the desired schedule.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p> </p>
<h2>Need help?<o:p></o:p></h2>
<p class="MsoNormal">If you need help implementing best practices for your vNICs,
the IBM i Technology Services team (formerly known as Lab Services) is
available to help with implementation planning, execution, and knowledge
transfer.<span style="mso-spacerun: yes;"> </span>See <a href="https://www.ibm.com/it-infrastructure/services/lab-services">https://www.ibm.com/it-infrastructure/services/lab-services</a>
for contact information or speak to your IBM Sales Representative or Business
Partner.<span style="mso-spacerun: yes;"> </span>If you are planning a new
hardware purchase, you can include implementation services by the Technology
Services team in your purchase.<o:p></o:p></p>
<h2>Disclaimer<o:p></o:p></h2>
<p class="MsoNormal">I am an employee of IBM on the IBM i Technology Services
team (formerly known as Lab Services).<span style="mso-spacerun: yes;"> </span>The
opinions in this post are my own and don't necessarily represent IBM's
positions, strategies, or opinions.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>References<o:p></o:p></h2>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Getting started with Open Source Package Management in IBM i
ACS<o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/getting-started-open-source-package-management-ibm-i-acs">https://www.ibm.com/support/pages/getting-started-open-source-package-management-ibm-i-acs</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM i ACS Open Source Package Management Auth Fail Error<o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/node/1167988">https://www.ibm.com/support/pages/node/1167988</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3547212460241842466.post-59593633210715156152023-01-08T10:53:00.002-07:002023-01-08T11:12:30.719-07:00Please Stop Changing Partition Profiles<p> </p><p class="MsoNormal"><br /></p>
<p class="MsoNormal">The Enhanced HMC interface is here to stay.<span style="mso-spacerun: yes;"> </span>If you are still changing partition profiles on
your Power HMC, you really need to start using the new functionality instead,
or you risk getting out of sync and losing changes.<span style="mso-spacerun: yes;"> </span>It is painful to create a bunch of new
virtual fiber channel adapters, and then have them magically disappear with
your next reboot.<span style="mso-spacerun: yes;"> </span>It’s even worse when
you reboot a VIOS and choose an out of date partition profile and suddenly some
of your client disks go away.<span style="mso-spacerun: yes;"> </span>Ask me how
I know.<o:p></o:p></p>
<p class="MsoNormal">I normally try to write articles focused on IBM i, but in
this case, there really isn’t any difference between IBM i, AIX, and Linux.<span style="mso-spacerun: yes;"> </span>All partitions (especially VIOS) should
follow the same rules.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>First a bit of history<o:p></o:p></h2>
<p class="MsoNormal">IBM made the Enhanced HMC interface available as an option
with version <span style="background: white; color: black; mso-color-alt: windowtext;">8.1.0.1.<span style="mso-spacerun: yes;"> </span>If you were an administrator like me, you
just looked at it once or twice, figured it didn’t make any sense compared to
what you were used to, and just selected “Classic” from the menu when you
logged in.<span style="mso-spacerun: yes;"> </span></span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">Version V8R8.7.0 officially eliminated the Classic interface,
but some enterprising users found and published a backdoor approach to access
the classic interface even at that level (see </span><a href="https://theibmi.org/2019/09/11/enable-classic-hmc-gui-on-release-v9r1/"><span face=""IBM Plex Sans",sans-serif" style="background: white;">Bart’s Blog -
Enable classic HMC GUI on release V9R1</span></a><span style="background: white; color: black; mso-color-alt: windowtext;"> – referenced below) That unofficial
approach was then shut down for good in May of 2020.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">Why?<span style="mso-spacerun: yes;"> </span>Because IBM is
focusing development on a single easy to use interface that leverages DLPAR
operations for all the new features like vNIC (see my previous blog post if
that’s new to you).</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span face=""IBM Plex Sans",sans-serif" style="background: white; color: #161616;"><o:p> </o:p></span></p>
<h2><span style="background: white;">Making the Move<o:p></o:p></span></h2>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">First and foremost, make sure that your partition profiles
are in sync with the running profile.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">There is an excellent </span><a href="https://community.ibm.com/community/user/power/blogs/hariganesh-muralidharan1/2020/06/08/sync-curr-config-and-inactive-lpar-config-change"><span face=""IBM Plex Sans",sans-serif" style="background: white;">blog post in
the IBM Community</span></a><span style="background: white; color: black; mso-color-alt: windowtext;"> that explains this in much more detail. </span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">If you are using VIOS, DON’T FORGET THE VIOS!<span style="mso-spacerun: yes;"> </span>There is far more risk of lost configuration
on VIOS than any other partition, because when you are using the Enhanced GUI,
you are often making dynamic changes to VIOS you may not even be aware of.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">The gist of it is that you should be running with the “Save
configuration changes to profile” setting at “Enabled”.<span style="mso-spacerun: yes;"> </span>If it is not currently set to enabled, you
need to get it set that way.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal">If the setting is currently “disabled”, start by saving your
current configuration to the default partition profile.<span style="mso-spacerun: yes;"> </span>Select the partition view for the desired
partition from the GUI, select Partition Actions->Profiles->Save Current
Configuration and select the default profile name.<span style="mso-spacerun: yes;"> </span>Most users only have one profile per
partition.<span style="mso-spacerun: yes;"> </span>If you are one of the few
that has more than one, pick a name for the profile that you will use from now
on. <span style="mso-spacerun: yes;"> </span>The default name used for newly
created partitions is “default_profile”, so that is pretty good choice for a
name. <span style="mso-spacerun: yes;"> </span>Save the configuration with the
desired name.<span style="mso-spacerun: yes;"> </span>If you created a new name,
go into “Manage Profiles” for your last time and change it your newly saved
profile as the default.<span style="mso-spacerun: yes;"> </span>Now is also a
good time to delete all those profiles you will not be using any more.<o:p></o:p></p>
<p class="MsoNormal">Now you can change the <span style="background: white; color: black; mso-color-alt: windowtext;">“Save configuration changes to profile” setting to
“Enabled”.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white;"><o:p> </o:p></span></p>
<h2><span style="background: white;">Doing it the Enhanced way<o:p></o:p></span></h2>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">Once you have this setting enabled, just stay away from
“Manage Profiles” and make all of your changes using the Enhanced GUI dynamic menu
operations available from the left-hand menu of the partition view.<span style="mso-spacerun: yes;"> </span></span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">When you need to activate a partition that you previously
shutdown, make sure you use the “Current Configuration” option rather than picking
a partition profile.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">The biggest difference between changing partition profiles
and restarting with a different profile is that in the Enhanced GUI, it will
make the changes dynamically on a running partition.<span style="mso-spacerun: yes;"> </span>It will also make the corresponding changes
on the VIOS, if necessary.<span style="mso-spacerun: yes;"> </span>The days of
keeping track of virtual port numbers can be gone, if you let them.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">You’ll find that when you Google the procedure to do anything
on the HMC, you will often find articles and screen shots that point you to modify
the profile.<span style="mso-spacerun: yes;"> </span>If at any point, one of
these articles suggests using the Manage Profiles option or tells you to select
a specific profile when activating a partition, keep looking for a new
procedure.<span style="mso-spacerun: yes;"> </span>You can often get good basic
information from these articles, but the specific procedures are likely to get
you into trouble.</span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">Enhanced HMC changes are typically dynamic on a running
partition.<span style="mso-spacerun: yes;"> </span>This requires communication
between the HMC and the running partition, which you will typically see
referred to as an RMC connection.<span style="mso-spacerun: yes;"> </span>One
difference for the IBM i world is that IBM i uses a LIC connection rather than
the RMC connections that are used by AIX and Linux.<span style="mso-spacerun: yes;"> </span>This all means that you won’t see an RMC
active flag on an IBM i partition.<span style="mso-spacerun: yes;"> </span>I
mention this for two reasons.<span style="mso-spacerun: yes;"> </span>First,
much of the documentation you will run into will mention the need for an active
RMC connection for various procedures.<span style="mso-spacerun: yes;"> </span>That
is not true for IBM i.<span style="mso-spacerun: yes;"> </span>Second, the O/S
on an IBM i does need to be operating to make some dynamic changes.<span style="mso-spacerun: yes;"> </span>The error message you’ll get while attempting
to make some changes on an activated IBM i partition with refer to RMC, but it
really means its not booted to a DLpar capable state.<span style="mso-spacerun: yes;"> </span></span><span style="background: white;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="background: white; color: black; mso-color-alt: windowtext;">You may notice that there are things you cannot change using
the Enhanced interface while the partition is active.<span style="mso-spacerun: yes;"> </span>Some examples are max processor, max memory,
max virtual adapters, and processor compatibility mode.<span style="mso-spacerun: yes;"> </span>All these options require a shutdown and
restart.<span style="mso-spacerun: yes;"> </span>You will be permitted to make
the changes while the partition is shutdown.</span><span style="background: white;"><o:p></o:p></span></p>
<h2><span style="background: white;">Why is it so slow? (Spoiler - it's not)</span><o:p></o:p></h2>
<p class="MsoNormal">You might not believe me here, but it isn’t slow.<span style="mso-spacerun: yes;"> </span>It just feels that way because it is doing
everything dynamically right now when you are used to delaying all that
processing to partition activation.<o:p></o:p></p>
<p class="MsoNormal">Making changes to profiles is blazing fast because they are
not actually changing any real resources, but you will pay the price during
activation of that profile.<span style="mso-spacerun: yes;"> </span>On the
contrary, when you make a change to a running partition with a dynamic HMC
change, all that processing that happens in the hypervisor and O/S to add that
resource will happen immediately -- while you wait.<span style="mso-spacerun: yes;"> </span>That’s right, while you wait means, well...
you will be waiting.<o:p></o:p></p>
<p class="MsoNormal">I’ve actually done some benchmarks on new system setups to
compare dynamic operations with HMC commands (chhwres - equivalent to the
Enhanced HMC GUI)<span style="mso-spacerun: yes;"> </span>to HMC profile change
commands (chsyscfg commands) that get applied via the “chsyscfg -o apply”
command.<span style="mso-spacerun: yes;"> </span>The chhwres commands on either
a running or inactive partition, tend to be slow to operate, while the
equivalent profile changes are very fast until they are either applied via apply
command or profile activation.<span style="mso-spacerun: yes;"> </span>In the
end, it comes down to <i>when</i> you are going to wait.<span style="mso-spacerun: yes;"> </span>You can wait now, or you can wait later, but
you are always going to wait for the actual resource creation in the
hypervisor.<o:p></o:p></p>
<p class="MsoNormal">To be completely honest, I’m a command line guy.<span style="mso-spacerun: yes;"> </span>Sure, I’ll use the HMC GUI to create small
test partitions and add a few virtual network or virtual fiber channel
connections when I must.<span style="mso-spacerun: yes;"> </span>I’m much more
likely to create a command script to do it all for anything more than a couple
resources.<span style="mso-spacerun: yes;"> </span>I don’t have the patience to
create hundreds of virtual fiber channel connections on a giant Power 1080 one
by one in a GUI.<span style="mso-spacerun: yes;"> </span>That said, most IBM i
admins don’t create a lot of resources except during hardware refreshes and
migrations, so using the GUI is right way to learn – it’s also safer.<o:p></o:p></p>
<p class="MsoNormal">I’ll post some more details of the command line way of
creating and configuring partitions and partition resources in the future for
those that are interested in that approach.<o:p></o:p></p>
<h2 style="text-align: left;">Need Help?</h2><p class="MsoNormal">If you need help fixing a profile problem, or with a hardware
refresh or migration and don’t want to go it alone, the IBM i Technology
Services team (formerly known as Lab Services) is available to help with
implementation planning, execution, and knowledge transfer.<span style="mso-spacerun: yes;"> </span>See <a href="https://www.ibm.com/it-infrastructure/services/lab-services">https://www.ibm.com/it-infrastructure/services/lab-services</a>
for contact information or speak to your IBM Sales Representative or Business
Partner.<span style="mso-spacerun: yes;"> </span>If you are planning a new
hardware purchase, you can include implementation services by the Technology
Services team in your purchase.<o:p></o:p></p>
<h2>Disclaimer<o:p></o:p></h2>
<p class="MsoNormal">I am an employee of IBM on the IBM i Technology Services
team (formerly known as Lab Services).<span style="mso-spacerun: yes;"> </span>The
opinions in this post are my own and don't necessarily represent IBM's
positions, strategies, or opinions.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>References<o:p></o:p></h2>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Synchronize Current Configuration and configuration change
on inactive partition in HMC Enhanced UI<o:p></o:p></p>
<p class="MsoNormal"><a href="https://community.ibm.com/community/user/power/blogs/hariganesh-muralidharan1/2020/06/08/sync-curr-config-and-inactive-lpar-config-change">https://community.ibm.com/community/user/power/blogs/hariganesh-muralidharan1/2020/06/08/sync-curr-config-and-inactive-lpar-config-change</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span face=""IBM Plex Sans",sans-serif" style="background: white; color: #161616;">Bart’s Blog - Enable classic HMC GUI on release
V9R1</span><o:p></o:p></p>
<p class="MsoNormal"><a href="https://theibmi.org/2019/09/11/enable-classic-hmc-gui-on-release-v9r1/"><span face=""IBM Plex Sans",sans-serif" style="background: white;">h</span>ttps://theibmi.org/2019/09/11/enable-classic-hmc-gui-on-release-v9r1/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM Support - Saving Configuration Changes To Profile<o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/saving-configuration-changes-profile">https://www.ibm.com/support/pages/saving-configuration-changes-profile</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">How to create Shared Ethernet Adapater without touching VIOS<o:p></o:p></p>
<p class="MsoNormal"><a href="https://theibmi.org/2016/03/26/how-to-create-sea-with-no-touch-vio/">https://theibmi.org/2016/03/26/how-to-create-sea-with-no-touch-vio/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">HMC – Enhanced+ interface tricks<o:p></o:p></p>
<p class="MsoNormal"><a href="https://theibmi.org/2020/11/15/hmc-enhanced-interface-tricks/">https://theibmi.org/2020/11/15/hmc-enhanced-interface-tricks/</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3547212460241842466.post-67161030573583122372022-12-15T09:58:00.000-07:002022-12-15T09:58:56.137-07:00PowerVM VIOS for IBM i<p> </p><p class="MsoNormal">In this post, I will discuss the pros and cons of creating a
completely virtualized IBM i environment with redundant VIOS (Virtual I/O Server).<span style="mso-spacerun: yes;"> </span>You can just look at the name of this website
to understand where I stand on that issue.<span style="mso-spacerun: yes;">
</span>Many IBM i administrators try to avoid VIOS for several reasons.<span style="mso-spacerun: yes;"> </span>To be completely honest, a LONG time ago I
was even one of them.<span style="mso-spacerun: yes;"> </span>That was a mistake.<span style="mso-spacerun: yes;"> </span>I want to make the case for why you should
consider VIOS for I/O virtualization on your next system.<o:p></o:p></p>
<p class="MsoNormal">In the present day, there are many options for virtualizing
the workload on an IBM Power server.<span style="mso-spacerun: yes;"> </span>The
options range from absolutely no virtualization (a non-partitioned system), to
all Input/Output and processor completely virtualized and mobile.<span style="mso-spacerun: yes;"> </span>According to the 2022 Fortra (formerly
HelpSystems) survey, 22% of you have a single partition, and 25% have two
partitions.<span style="mso-spacerun: yes;"> </span>If that’s you, you probably
don’t need VIOS... yet.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">It is also common to find particularly critical partitions
with dedicated processors and dedicated I/O resources on the same Power servers
as fully virtualized partitions that are sharing resources.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">I’m a big fan of virtualizing everything, but I understand
that is not always optimal.<span style="mso-spacerun: yes;"> </span>Fortunately,
PowerVM has the flexibility provide the right choice for you on a
partition-by-partition basis.<o:p></o:p></p>
<h2>Why should you virtualize I/O?<span style="mso-spacerun: yes;"> </span><o:p></o:p></h2>
<p class="MsoNormal">Ask yourself a question:<span style="mso-spacerun: yes;">
</span>If you have more than one partition, why don’t you buy a separate Power
system for each partition?<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">Your business probably requires multiple partitions for a
reason: workload splitting, different applications, development/testing
environments, etc.<span style="mso-spacerun: yes;"> </span>You also have good
reasons to consolidate your separate workloads onto a smaller number of more
powerful systems.<span style="mso-spacerun: yes;"> </span>Usually, those reasons
relate to things like cost, allowance for growth, limited floor space, power,
or cooling requirements.<o:p></o:p></p>
<p class="MsoNormal">The same reasons apply to why you should virtualize your I/O
resources.<span style="mso-spacerun: yes;"> </span>Ethernet infrastructure
(especially 10G) is a limited resource.<span style="mso-spacerun: yes;">
</span>Switches, cabling and SFPs all add to expenses and complexity.<o:p></o:p></p>
<p class="MsoNormal">Sharing fiber channel ports for storage also reduces the
number of ports needed on SAN switches, as well as reducing cable needs.<span style="mso-spacerun: yes;"> </span>This saves money and time.<o:p></o:p></p>
<p class="MsoNormal">If you use external (SAN) storage, you can even use Live
Partition Mobility (LPM) to move <i>running </i>partitions between physical
servers.<span style="mso-spacerun: yes;"> </span>This is a very common practice
in the AIX world, but fairly rare for IBM i.<span style="mso-spacerun: yes;">
</span>More to come on that.<o:p></o:p></p>
<p class="MsoNormal">External Storage also allows you to leverage technologies
such as FlashCopy to create backups with almost zero downtime or create test or
reporting copies practically instantly.<span style="mso-spacerun: yes;">
</span>It will also greatly simplify server migrations and enable storage-based
replication for High Availability and Disaster Recovery solutions.<span style="mso-spacerun: yes;"> </span>I’ll write a future article that delves
deeper into the benefits of external storage, as it is a technology that
deserves a deep dive.<o:p></o:p></p>
<p class="MsoNormal">When you have a fully virtualized PowerVM infrastructure in
place, creating a new partition becomes a very simple thing.<span style="mso-spacerun: yes;"> </span>There is no longer any need to assign any
physical resources.<span style="mso-spacerun: yes;"> </span>Just create new
virtual resources with the HMC GUI and your partition (IBM i, AIX, or Linux) is
ready to go.<span style="mso-spacerun: yes;"> </span>Okay, you might need to do
some zoning and maybe assign some storage before you can use it, but the
partition will be ready to go.<o:p></o:p></p>
<h2>Redundancy is critical<o:p></o:p></h2>
<p class="MsoNormal">Proper virtualization leverages redundancy to improve
reliability.<span style="mso-spacerun: yes;"> </span>Ideally, all your
virtualized resources should have backup.<span style="mso-spacerun: yes;">
</span><o:p></o:p></p>
<p class="MsoNormal">Virtual Ethernet connections should be based on vNIC with
multiple backing adapters for automatic failover, or Shared Ethernet Adapters
backed by multiple physical adapters in multiple VIOS.<span style="mso-spacerun: yes;"> </span>Each adapter should connect to the network
via separate network switches.<span style="mso-spacerun: yes;"> </span>Eliminate
all single points of failure and you will eliminate many potential problems
before they happen.<o:p></o:p></p>
<p class="MsoNormal">Storage should have multiple paths via multiple fiber
channel cards owned by multiple VIOS partitions connected through multiple SAN
switches (fabrics) to multiple storage ports.<span style="mso-spacerun: yes;">
</span>Again, eliminate those single points of failure.<o:p></o:p></p>
<p class="MsoNormal">A properly implemented virtual infrastructure is more
reliable than individual physical adapters directly mapped to partitions.<o:p></o:p></p>
<h2>Don’t fear the VIOS<o:p></o:p></h2>
<p class="MsoNormal">If I had any musical talent, I’d make a version of the
classic “Don’t Fear the Reaper” song as “Don’t Fear the VIOS”.<span style="mso-spacerun: yes;"> </span>I don’t, so I’ll just stick with text.<span style="mso-spacerun: yes;"> </span>Trust me.<span style="mso-spacerun: yes;">
</span>It’s better this way.<o:p></o:p></p>
<p class="MsoNormal">Many IBM i administrators want to avoid VIOS because it is
based on AIX, which is an unfamiliar technology.<span style="mso-spacerun: yes;"> </span>As I mentioned before, I was one of those
until I spent a few years at a company which used VIOS extensively.<o:p></o:p></p>
<p class="MsoNormal">Let me be very clear about this.<span style="mso-spacerun: yes;"> </span>AIX guys are NOT smarter than IBM i
guys.<span style="mso-spacerun: yes;"> </span>They just understand a different
command syntax.<span style="mso-spacerun: yes;"> </span>They might be smarter
than Windows guys, but who isn’t, right?<o:p></o:p></p>
<p class="MsoNormal">AIX users should NOT be the only ones that benefit from VIOS
in their environments.<span style="mso-spacerun: yes;"> </span>VIOS is intended
to be implemented as an appliance, similar to the HMC, but exclusively in
software.<span style="mso-spacerun: yes;"> </span>There is a connection to the
HMC that is the primary means of configuration.<span style="mso-spacerun: yes;">
</span>There is also a command line environment that is subset of simplified
AIX commands and some commands that are specific to VIOS.<span style="mso-spacerun: yes;"> </span>It is well documented with both online help
and manuals, but you will rarely need to use it.<o:p></o:p></p>
<p class="MsoNormal">The fact is, once you have done the basic install of VIOS, all
your ongoing monitoring and configuration can be completed from the modern Enhanced
HMC GUI interface.<span style="mso-spacerun: yes;"> </span>If you want to add a partition,
map a new fiber channel port , configure a new vNIC, etc. You do it all with
clicks on a web interface.<span style="mso-spacerun: yes;"> </span>The only time
you MUST use the command line on the VIOS is for a few commands during an install,
and to install software updates.<span style="mso-spacerun: yes;">
</span>Software updates are usually a painless process that involves an install
to an alternate boot disk and a simple reboot to activate.<span style="mso-spacerun: yes;"> </span>The alternate disk install also means the
upgrades are completely reversible in case of problems.<span style="mso-spacerun: yes;"> </span>Remember that you want to have redundant
connections to multiple VIOS, so that reboot will not be disruptive to your
environment.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">I should mention that just because you usually don’t have to
use the command line interface doesn’t mean you won’t want to use the command
line interface.<span style="mso-spacerun: yes;"> </span>There is a massive
amount of information to be had from those simple commands.<span style="mso-spacerun: yes;"> </span>Watch for a future post where I publish and <i>explain
</i>some of my favorite information gathering VIOS commands.<o:p></o:p></p>
<p class="MsoNormal">The benefits of VIOS outweigh the costs, especially if you
are using external storage.<o:p></o:p></p>
<h2>Licensing topics<o:p></o:p></h2>
<p class="MsoNormal">Fun fact, you are probably already licensed for VIOS.<span style="mso-spacerun: yes;"> </span>PowerVM is required for partitioning, and all
editions include VIOS.<span style="mso-spacerun: yes;"> </span>If have PowerVM
licenses for your server, you are already entitled to install VIOS.<span style="mso-spacerun: yes;"> </span>You can get it from IBM Entitled System
Support by going to “My Entitled Software”, “By Product” and select
5765-VE3.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">Another important consideration for those of you with extra
processors not licensed for IBM i, VIOS is not IBM i, so you do not need those
licenses for the processors running VIOS.<span style="mso-spacerun: yes;">
</span>That means the processor overhead related to handling the I/O
virtualization does not have a premium beyond the cost to activate the
processor.<span style="mso-spacerun: yes;"> </span>You can make sure you are in
compliance by using HMC processor pools to limit the IBM i partitions to the
number of licensed processors, and putting your VIOS (and Linux) in an uncapped
pool.<o:p></o:p></p>
<p class="MsoNormal">Another virtualization topic specific to IBM i is the way
the O/S and most applications are licensed.<span style="mso-spacerun: yes;">
</span>I mentioned earlier that Live Partition Mobility, moving a running partition
to a different server, is a common practice for AIX shops.<span style="mso-spacerun: yes;"> </span>It is pretty rare for IBM i.<span style="mso-spacerun: yes;"> </span>I think one of the key reasons that has been
true historically is that AIX O/S and applications are not generally licensed
to a processor while IBM i O/S and applications are pretty much always licensed
to a processor serial number.<span style="mso-spacerun: yes;"> </span>That means
moving an active IBM i partition to another Power server can result in license
problems.<span style="mso-spacerun: yes;"> </span>Fortunately, IBM recently
announced Virtual Serial Numbers that can be attached to a partition and
migrate with it.<span style="mso-spacerun: yes;"> </span>If Live Partition
Mobility appeals to you, look into getting a Virtual Serial Number.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">I should mention that since LPM moves memory over a network
to the other server, LPM on IBM i may require a much more robust network
environment than the equivalent AIX resources.<span style="mso-spacerun: yes;">
</span>IBM i uses single level storage, so it uses large amounts of very active
memory.<span style="mso-spacerun: yes;"> </span>There are certainly memory size
and activity limits that could preclude the use of LPM for very large
environments.<span style="mso-spacerun: yes;"> </span>As always, your
environment matters, and your results may vary.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>iVirtualization (AKA i hosting i)<o:p></o:p></h2>
<p class="MsoNormal">There is another option for virtualizing I/O and disk
resources for a client partition by using the iVirtualization functionality
built into IBM i since V6.1.<span style="mso-spacerun: yes;"> </span>This
functionality allows you to virtualize ethernet adapters owned by the parent
partition and to create virtual disk objects that are shared to another client
partition as virtual SCSI disks.<o:p></o:p></p>
<p class="MsoNormal">The *NWS* commands to support this are all native IBM
commands that will look familiar to IBM i administrators.<span style="mso-spacerun: yes;"> </span>Don’t kid yourself.<span style="mso-spacerun: yes;"> </span>They are no less complex than the
corresponding VIOS commands to someone that has never used them.<o:p></o:p></p>
<p class="MsoNormal">In some limited situations, iVirtualization might be a
viable option.<span style="mso-spacerun: yes;"> </span>For example, on a small
system with internal NVMe on a single backplane such that it is not possible to
split between multiple VIOS for redundancy.<span style="mso-spacerun: yes;">
</span><o:p></o:p></p>
<p class="MsoNormal">Another case where iVirtualization might be preferred is for
a small linux test partition hosted from an existing IBM i partition with
internal disk and no VIOS infrastructure.<o:p></o:p></p>
<p class="MsoNormal">I would not use it with external storage in any case as it
would lose all of the benefit of multipathing.<o:p></o:p></p>
<p class="MsoNormal">Now here are the primary reasons I would recommend VIOS over
iVirtualization:<o:p></o:p></p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>License costs.</b><span style="mso-spacerun: yes;"> </span>Hosting on IBM i means paying for an IBM i
license for work that could be free.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Performance.</b><span style="mso-spacerun: yes;"> </span>The numbers I have seen have consistently
shown the client partitions do not perform as well as an equivalent VIOS
configuration.<span style="mso-spacerun: yes;"> </span>This is especially
problematic with an IBM i client as performance is related to number of disks,
which results in more objects and more overhead.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Completely manual configuration.</b><span style="mso-spacerun: yes;"> </span>The HMC GUI configuration that is available
with VIOS does not work with iVirtualization, so it needs to be configured
completely with commands.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-weight: bold; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>No Redundancy.<span style="mso-spacerun: yes;"> </span></b>When the host is down, the clients are
down. To be fair, you could use multiple host partitions and mirror disks in
the client, but you can do that with VIOS also.<b><o:p></o:p></b></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-weight: bold; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>No LPM.</b><span style="mso-spacerun: yes;">
</span>Live Partition Mobility is not supported for clients of iVirtualization.<b><o:p></o:p></b></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-bidi-font-weight: bold; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b>No development.</b><span style="mso-spacerun: yes;"> </span>If you look at the table of changes in the IBM
i Virtualization Summary referenced below, you will see that there has been
only one change to iVirtualization since 2015, compared to constant development
and improvements for VIOS.<b><o:p></o:p></b></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>What if you need help implementing VIOS with IBM i?<o:p></o:p></h2>
<p class="MsoNormal">Whether you have a large environment or small, implementing
new technologies can be challenging.<span style="mso-spacerun: yes;"> </span>If
you need help beyond the available documentation, the IBM i Technology Services
team (formerly known as Lab Services) is available to help with implementation
planning, execution, and knowledge transfer.<span style="mso-spacerun: yes;">
</span>See <a href="https://www.ibm.com/it-infrastructure/services/lab-services">https://www.ibm.com/it-infrastructure/services/lab-services</a>
for contact information or speak to your IBM Sales Representative or Business
Partner.<span style="mso-spacerun: yes;"> </span>If you are planning a new
hardware purchase, you can include implementation services by the Technology
Services team in your purchase.<o:p></o:p></p>
<h2>Disclaimer<o:p></o:p></h2>
<p class="MsoNormal">I am an employee of IBM on the IBM i Technology Services
team (formerly known as Lab Services).<span style="mso-spacerun: yes;"> </span>The
opinions in this post are mine and don't necessarily represent IBM's positions,
strategies, or opinions.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">References:<o:p></o:p></p>
<p class="MsoNormal">2022 IBM i Marketplace Survey Results - Fortra<o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.fortra.com/resources/guides/ibm-i-marketplace-survey-results">https://www.fortra.com/resources/guides/ibm-i-marketplace-survey-results</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM i Virtualization Summary <o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/node/1135420">https://www.ibm.com/support/pages/node/1135420</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3547212460241842466.post-14115681389878207672022-11-08T10:12:00.059-07:002022-11-15T19:34:50.428-07:00Introduction to SR-IOV and vNIC for IBM i<p> </p><p class="MsoNormal">This is the first in a series of articles on frequently
overlooked Power systems features that highlight the value for IBM i customers,
starting with sharing Ethernet adapters with SR-IOV, and the added benefits
that can be achieved with vNIC technology on top of SR-IOV.<span style="mso-spacerun: yes;"> </span><o:p></o:p></p>
<p class="MsoNormal">Whether you have an existing system that is already capable
of these features, or you are considering migrating to new hardware, you can
only benefit from knowing what your options are.<o:p></o:p></p>
<h2>What Is SR-IOV?<o:p></o:p></h2>
<p class="MsoNormal">SR-IOV (Single Root Input/Output Virtualization) is a hardware
specification that allows multiple operating systems to simultaneously use a
single I/O adapter in a virtualized environment.<span style="mso-spacerun: yes;"> </span>It is not unique to the Power Hypervisor
(PHYP).<span style="mso-spacerun: yes;"> </span>You can find SR-IOV being used
heavily in x86 based virtualization, such as VMWare or Hyper-V – a fact that
just serves to complicate searches for information related to IBM i.<o:p></o:p></p>
<p class="MsoNormal">More to the point for the IBM i administrator, it allows a
single SR-IOV capable adapter to be shared by multiple LPARs.<span style="mso-spacerun: yes;"> </span>You can split a single adapter with two
ports, dedicating each port to a separate lpar, or you can go more granular and
share different percentages of the bandwidth of a single physical port between
multiple partitions.<span style="mso-spacerun: yes;"> </span>When sharing a
single physical port, you get to specify the minimum percentage of outgoing
bandwidth each partition gets, allowing each partition to use available
bandwidth to burst higher when necessary.<span style="mso-spacerun: yes;">
</span>It is also possible to limit the maximum outgoing bandwidth a given
partition will use, although this is only possible via the HMC CLI, not the HMC
GUI.<o:p></o:p></p>
<h2>What is vNIC?<o:p></o:p></h2>
<p class="MsoNormal">vNIC is a Power virtualization technology built into PowerVM
that leverages the combination of VIOS (Virtual I/O Server) virtualization with
SR-IOV adapters to get the performance and flexibility of SR-IOV with the
additional flexibility and redundancy of a fully virtualized solution.<span style="mso-spacerun: yes;"> </span>I expect to expand on VIOS in much more
detail in future article.<span style="mso-spacerun: yes;"> </span>For now, I’ll
just say that vNIC provides an automated active/passive failover ability and supports the use
of Live Partition Mobility.<span style="mso-spacerun: yes;"> </span>If you
already use VIOS, you should strongly consider SR-IOV adapters with vNIC rather
than Shared Ethernet Adapters (SEA) unless you need the active/active load sharing configuration that is only available with SEA.<span style="mso-spacerun: yes;"> </span>If
you don’t use VIOS, watch out for a future article for why you should.<o:p></o:p></p>
<h2>Why SR-IOV?<o:p></o:p></h2>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Better use of limited resources.</b><span style="mso-spacerun: yes;"> </span>10G ethernet adapters have become common in
enterprise configurations.<span style="mso-spacerun: yes;"> </span>Most of these
adapters have multiple ports.<span style="mso-spacerun: yes;"> </span>Without SR-IOV,
each adapter is usually dedicated to a single partition, often leaving the
extra ports unused while additional adapters are dedicated to other partitions,
leaving even more ports unused.<span style="mso-spacerun: yes;"> </span>How many
of these ports are utilized to their full capacity?<span style="mso-spacerun: yes;"> </span>Not as many as you might think (seriously,
collect some performance stats and see for yourself).<span style="mso-spacerun: yes;"> </span>More adapters used at a fraction of their
capacity means more cabling and more network switch ports, all used at a
fraction of their capacity.<span style="mso-spacerun: yes;"> </span>That gets
costly, for both the server and network budgets, especially when working with
10G ports.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]--><b>More flexibility.</b><span style="mso-spacerun: yes;"> </span>Once you have connected ports to network
switches, you can add partitions that use those ports without any additional
cabling or network configuration.<span style="mso-spacerun: yes;"> </span>This
is especially true if you configure those ports as trunks and use VLAN tagging
at the IBM i TCP/IP configuration to access different networks and IP address
ranges.<o:p></o:p></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Better Performance than other shared
configurations.</b><span style="mso-spacerun: yes;"> </span>Compared to
traditional server-based networking configurations (VIOS Shared Ethernet
Adapters or IBM i NWS Virtual Ethernet), SR-IOV connections perform much
better.<span style="mso-spacerun: yes;"> </span>Virtual ethernet connections
have processor overhead, and many tuning parameters that limit performance.<span style="mso-spacerun: yes;"> </span>SR-IOV establishes a hypervisor managed path
to the hardware that is second only to a dedicated adapter.<span style="mso-spacerun: yes;"> </span>In the real world, SR-IOV will perform
effectively the same as a dedicated adapter, and better than any server
virtualized adapter.<o:p></o:p></p>
<h2>Who should use SR-IOV?<o:p></o:p></h2>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Large Enterprises</b> should consider SR-IOV
and vNIC technology to achieve high bandwidth connectivity to enterprise scale
10G (and up) infrastructure .<span style="mso-spacerun: yes;"> </span>Automatic
failover (vNIC) to redundant connections ensures connectivity that leverages
the highly redundant network infrastructures that exist in high-end
enterprises. <o:p></o:p></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]--><b>Small businesses</b> should consider SR-IOV
and vNIC technology to get the maximum capacity out of the investment in
network connectivity.<span style="mso-spacerun: yes;"> </span>Fewer adapters,
less cabling and a smaller number of network ports is easier on the budget,
while still providing the ability to adapt to changing business needs.<span style="mso-spacerun: yes;"> </span>SR-IOV adapters provide the ability to share
adapters between partitions without any server based virtualization, resulting
in a simple to maintain shared configuration when other virtualization
functions are not required.<o:p></o:p></p>
<h2>What else do I need to know?<o:p></o:p></h2>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><!--[if !supportLists]--><span style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7pt "Times New Roman";">
</span></span></span><!--[endif]-->For all of the following, see the SR-IOV FAQ for
details.<span style="mso-spacerun: yes;"> </span>It can be found at:<span style="mso-spacerun: yes;"> </span><a href="https://community.ibm.com/community/user/power/viewdocument/sr-iov-vnic-and-hnv-information">https://community.ibm.com/community/user/power/viewdocument/sr-iov-vnic-and-hnv-information</a><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"></p><ul style="text-align: left;"><li>You must have an SR-IOV supported adapter, so
make sure your IBM Sales Representative or Business Partner knows you want SR-IOV
when ordering a new system.</li><li>SR-IOV adapters must be placed in specific slots. On Power 9 and Power 10 hardware, this includes most of the slots in the system.</li><li>There are limits on the number of SR-IOV enabled
adapters per system.<span style="mso-spacerun: yes;"> </span>As of November 2022,
the maximum number of SR-IOV shared adapters is lower of 32 or the number of SR-IOV
slots in the system. This is not really limiting for most customers.</li><li>There are limits on how many shared (logical)
ports can be assigned to a physical port, depending on the specific adapter
(Ranging from 4 to 60)</li><li>There are limits on how many shared (logical)
ports can be assigned per adapter (ranging from 48 to 120)</li><li>SR-IOV adapters in shared mode require
Hypervisor memory (see FAQ)</li><li>Pay particular attention to limitations for 1G
ports on supported adapters, especially 1G SFP+ in 10G+ adapters as these may
not be supported for SR-IOV.</li><li>HMC is required for SR-IOV support. </li><li>VIOS is required for vNIC.<span style="mso-spacerun: yes;"> </span>VIOS is NOT required for SR-IOV.</li><li>Sharing a Link Aggregation (e.g. LACP) of
multiple ports is not allowed.<span style="mso-spacerun: yes;"> </span>This is
not as bad as it sounds as Link aggregation is effectively used as a redundancy
measure in a VIOS SEA configuration rather than as a performance measure.<span style="mso-spacerun: yes;"> </span>SEA simply does not have the capacity to use
more than a single link’s bandwidth.<span style="mso-spacerun: yes;"> </span>In
practically all cases where Link Aggregation is used with VIOS, vNIC with
failover is a better solution.<span style="mso-spacerun: yes;"> </span>In the
rare case that it is necessary, Link Aggregation can be managed at the IBM i O/S
level with the CRTLINETH RSRCNAME(*AGG) command if the SR-IOV physical ports
are 100% dedicated to a single partition.<span style="mso-spacerun: yes;">
</span>See <a href="https://www.ibm.com/support/pages/configuring-ethernet-link-aggregation">https://www.ibm.com/support/pages/configuring-ethernet-link-aggregation</a></li><li>Changing the minimum capacity of a SR-IOV
logical port is disruptive, so plan accordingly.<span style="mso-spacerun: yes;"> </span>Remember that the value is a minimum, and all
logical ports can burst higher.<span style="mso-spacerun: yes;"> </span>This
means that barring any specific continuous outgoing bandwidth requirements, you
are better off estimating low.</li><li>Bandwidth splitting on SR-IOV adapters is based
on outgoing bandwidth only.<span style="mso-spacerun: yes;"> </span>There is no
way to split incoming bandwidth, so consideration should be given to
anticipated incoming bandwidth when deciding on how many partitions can share a
port.</li><li>SR-IOV cards are not owned by any partition, so typically adapter firmware updates are included in System firmware updates. If necessary, there is a separate procedure to install adapter firmware updates separately that you may need to use. </li></ul><!--[if !supportLists]--><o:p></o:p><p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -0.25in;"><o:p></o:p></p>
<h2>How to configure an SR-IOV port on IBM i<o:p></o:p></h2>
<p class="MsoNormal">Rather than including a bunch of HMC screenshots that
duplicate existing resources, I’ll direct you to the excellent reference
material in the “Selected References” below, especially the <a href="https://www.redbooks.ibm.com/abstracts/redp5065.html" target="_blank">Redpaper</a>.<span style="mso-spacerun: yes;"> </span>These references will show you how to put an
SR-IOV adapter in shared or hypervisor mode and how to configure a logical port
for a partition.<span style="mso-spacerun: yes;"> </span>There is no difference
between doing this for AIX and IBM i.<span style="mso-spacerun: yes;">
</span>The specific web interface may change a bit with each HMC release, but
the concepts remain the same.<o:p></o:p></p>
<p class="MsoNormal">Once the resource is created, the easiest way to determine
the resource name is to select the partition from the HMC and get the CMNxx
resource name from the “Hardware Virtualized I/O” page for SR-IOV, or the
“vNIC” page for a vNIC.<span style="mso-spacerun: yes;"> </span>It will also
show up along with all of the other resources in WRKHDWRSC *CMN, or
STRSST.<span style="mso-spacerun: yes;"> </span>Once the resource name is
located, configure it exactly as you would any other Ethernet resource by
creating a Line description, IP address, etc.<o:p></o:p></p>
<p class="MsoNormal">You can dynamically add and remove SR-IOV and vNIC resources
to/from a running partition.<span style="mso-spacerun: yes;"> </span>Make sure that if you remove one, there are not any configurations using that resource.<o:p></o:p></p>
<h2>What if you need help implementing SR-IOV or vNIC on an IBM i?<o:p></o:p></h2>
<p class="MsoNormal">Whether you have a large environment or small, implementing
new technologies can be challenging.<span style="mso-spacerun: yes;"> </span>If
you need help beyond the available documentation, the IBM i Technology Services
team (formerly known as Lab Services) is available to help with implementation
planning, execution, and knowledge transfer.<span style="mso-spacerun: yes;">
</span>See <a href="https://www.ibm.com/it-infrastructure/services/lab-services">https://www.ibm.com/it-infrastructure/services/lab-services</a>
for contact information or speak to your IBM Sales Representative or Business
Partner.<span style="mso-spacerun: yes;"> </span>If you are planning a new
hardware purchase, you can include implementation services by the Technology
Services team in your purchase.<o:p></o:p></p>
<h2>Disclaimer<o:p></o:p></h2>
<p class="MsoNormal">I am an employee of IBM on the IBM i Technology Services
team (formerly known as Lab Services).<span style="mso-spacerun: yes;"> </span>The
opinions in this post are my own and don't necessarily represent IBM's
positions, strategies, or opinions.<o:p></o:p></p>
<h2>Selected References<o:p></o:p></h2>
<p class="MsoNormal">I often find that researching topics related to Power
Systems provides a wealth of information relating to AIX and VIOS, and
substantially less that relates directly to IBM i.<span style="mso-spacerun: yes;"> </span>Having spent a few years administering AIX
systems, I am familiar with the many excellent AIX blogs that are
available.<span style="mso-spacerun: yes;"> </span>Many of these references are
very AIX focused, but don’t let that dissuade you from reading them -- they are
also excellent resources for IBM i administrators.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM Power Systems SR-IOV Technical Overview and Introduction
Redpaper <a href="https://www.redbooks.ibm.com/abstracts/redp5065.html">https://www.redbooks.ibm.com/abstracts/redp5065.html</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM Support: Configuring Ethernet Link Aggregation <o:p></o:p></p>
<p class="MsoNormal"><a href="https://www.ibm.com/support/pages/configuring-ethernet-link-aggregation">https://www.ibm.com/support/pages/configuring-ethernet-link-aggregation</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">IBM Community: SR-IOV FAQ<o:p></o:p></p>
<p class="MsoNormal"><a href="https://community.ibm.com/community/user/power/viewdocument/sr-iov-vnic-and-hnv-information">https://community.ibm.com/community/user/power/viewdocument/sr-iov-vnic-and-hnv-information</a><span class="MsoHyperlink"><o:p></o:p></span></p>
<p class="MsoNormal"><span class="MsoHyperlink"><o:p><span style="text-decoration: none;"> </span></o:p></span></p>
<p class="MsoNormal">AIX for System Administrators – SR-IOV & vNIC summary
pages<o:p></o:p></p>
<p class="MsoNormal"><a href="http://aix4admins.blogspot.com/2016/01/sr-iov-vnic.html">http://aix4admins.blogspot.com/2016/01/sr-iov-vnic.html</a><o:p></o:p></p>
<p class="MsoNormal"><a href="http://aix4admins.blogspot.com/2017/03/vnic_20.html">http://aix4admins.blogspot.com/2017/03/vnic_20.html</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">YouTube – <span face=""Arial",sans-serif" style="color: #0f0f0f; font-size: 10.5pt; line-height: 107%;">This is the replay from the May
28th, Power Systems Virtual User Group Webinar covering Single Root I/O
Virtualization (SR-IOV) presented by expert Chuck Graham<o:p></o:p></span></p>
<p class="MsoNormal"><a href="https://youtu.be/1ANyxQaSXOI">https://youtu.be/1ANyxQaSXOI</a><o:p></o:p></p><p class="MsoNormal"><br /></p><h2 style="text-align: left;">TL;DR </h2><p style="text-align: left;">SR-IOV lets you share ethernet adapter cards across multiple IBM i partitions without using VIOS. vNIC adds the ability to include automatic active/passive failover if you also use VIOS.</p>
<p class="MsoNormal"><o:p> </o:p></p>Unknownnoreply@blogger.com2