vSphere 4.x Web Console – Missing VM’s

I’ve had difficulty finding this in the VMware forums and thought I’d post something on it here.

You may find that the vSphere web console is missing some VM’s. But, in the regular vSphere client, they are all there and accessible.

Chances are there’s a folder that was renamed or deleted.

In the vSphere client, select the VMs and Templates view. Expand your datacenter icon. You should have 1 folder. I believe the name should beDiscovered virtual machine“. If not, create one. Don’t do anything else. Don’t place any VM’s in that folder. Just leave it as is.

Wait several minutes and log into your vSphere web console. You should give vCenter some time to rebuild that list. If you launch the vSphere web console and the VM’s still aren’t there. Log off vSphere web console and give it some more time. Log back in and refresh if necessary.

You should be good to go after that.

If you don’t use the vSphere web console, then you don’t need that folder under the VMs and Templates view.

Advertisements

Apples and Oranges – Always More than one way

Many moons ago, I started modifying a script called “Healthcheck” written by Richard Giles (at least the source I found).

It was quite handy and gathered lots of useful tidbits of information. But, as our VMware environment grew, the script slowly began to take longer each time. We now have well over 250 virtual machines.

One “child” script I ended up with you’ll see next. This basically uses the same method to mine the information as did the Heathcheck PowerShell script. Against over 250 virtual machines, it was taking up to 45 minutest to complete at times.

Add-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue

$VMReportFile = Read-Host "Enter CSV Filename UNC PATH (i.e., \\machine\share\filename.csv)"
$VMInfoReport = @()
$VCServer = Connect-VIServer "vcenter"

Get-VM | ForEach-Object {
$vm = Get-View $_.ID
$row = "" | Select VMname, Host, Description
$row.VMname = $vm.Name
$row.Host = Get-VMHost -vm $vm.name
$row.Description = $vm.Config.Annotation
$VMInfoReport += $row
}

$VMInfoReport | sort -property VMname | Export-Csv $VMDataStoreFile -NoTypeInformation

# ---------------------------------------------------------------------------

# Disconnected from vCenter
Disconnect-VIServer $VCServer -Confirm:$False

trap{
Write-Host "ERROR: script execution was terminated.`n" $_.Exception.Message
# Disconnected from vCenter
Disconnect-VIServer $VCServer -Confirm:$False
break
}

Now, the weekness in this approach is having to do a get-view then query against vcenter to find the virtual machine’s host:

Get-VMHost -vm $vm.name

When I was trying to break the script down to understand this better, I used measure-task against various portions of the script. The culprit was always get get-vmhost:

measure-command {Get-VMHost -vm $vm.name}

This produced times ranging from 10-18 seconds.

I quickly realized the error of my ways in re-using the Healthcheck script. Since I only needed three items of information, I didn’t need to use get-view at all. I only needed to use get-vm.

Here is the script I ended up with shortly after:

Add-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue

$VMReportFile = Read-Host "Enter CSV Filename UNC PATH (i.e., \\machine\share\filename.csv)"
$VCServer = Connect-VIServer "vcenter"
$VMInfoReport = @()

$VMs = Get-VM | Out-Null

foreach ($item in $VMs){
$row = "" | Select VMname, Host, Description
$row.VMname = $item.Name
$row.Host = $item.Host
$row.Description = $item.Description
$VMInfoReport += $row
}

$VMInfoReport | sort -property VMname | Export-Csv $VMReportFile -NoTypeInformation

# ---------------------------------------------------------------------------

# Disconnected from vCenter
Disconnect-VIServer $VCServer -Confirm:$False

trap{
Write-Host "ERROR: script execution was terminated.`n" $_.Exception.Message
# Disconnected from vCenter
Disconnect-VIServer $VCServer -Confirm:$False
break
}

Now the script runs and I have my data in less than a minute.

Power Me On – Simple VMware PowerShell Script

I’ve got PowerShell open almost all the time.  But I don’t always have the vSphere client open.

At my company, we are also encouraging others to use PowerShell.  One quick idea I came up with is a simple script with a bit of simple error-handling that will allow anyone with the correct permissions to power on their VM when it’s in a powered-off state.

When we get a request to power on a machine, it’s just easy to run the script, type in the VM name when prompted, and voila!  Houston, we have power!

A similar script can also be written to power off, shut down, or restart. However, one might think “why go through all that trouble just to shut down a VM?  Why not just use

Shutdown-VmGuest "vmname"

Well, one reason would be if it’s a “one off” task.  It’s more convenient to do in a saved script.  It also allows for some error handling.  For instance, if a shut down script was being shared with a help desk staff you could also add scripting to confirm the VM name prior to shut down.

At any rate, here’s the power on script:

Add-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue

# Variables
$VCServer = "vcenter.myord.local"
$datacenter = "DC"

$PowerOnVM = Read-Host "Virtual Machine name you wish to power on?"
# Connect to vCenter
Connect-VIServer $VCServer | Out-Null

# Make sure VM exists
$VM = Get-VM | where-object {$_.Name -eq $PowerOnVM}

# If no value returned, tell user they provided wrong name
if ($VM -eq $null){
Write-Warning "$PowerOnVM is an invalid VM name"
break
}

# If it's already powered on, tell the user and abort
if ($VM.PowerState -eq "PoweredOn"){
Write-Host "$PowerOnVM is already powered on"
# Disconnected from vCenter
Disconnect-VIServer -Confirm:$False
break
}

# Beam me up Scotty, power on
(Get-View $VM.Id).PowerOnVM($_.Runtime.Host)

# Always Disconnected from vCenter so all connections are
#closed from your machine at the end of your script
Disconnect-VIServer -Confirm:$False

trap{
Write-Host "ERROR: script execution was terminated.`n" $_.Exception.Message
# Disconnected from vCenter
Disconnect-VIServer -Confirm:$False
break
}

PowerShell with VMware – TEAC Firmware Issues

My company has been investing resources in moving toward VMware. It has been a very good learning experience and a huge eye-opener into the world of virtualization. We have been re-using existing systems and implementing them into a decent sized VMware farm.

Early on, we began experiencing periodic guest disconnect issues. 99% of our guests are all Windows operating systems. Since we are still early in the implementation of this project we have not implemented our preferred backup solution (VCB). So, we continue to back up many guests via NetBackup. We attributed the cause of the disconnect issues to backup jobs causing bottle necks.

Our internal customers would call and describe the symptom where their remote desktop sessions would timeout, then a few seconds later, reconnect. SQL queries would time out as well. Again, because this was early in our implementation of VMware, we assumed it was a limited issue and attributed it to conflicts with backups. Only a select few guests experienced this peculiar problem.

A good number of weeks went by and we had over 10 hosts up and running with guest vm’s. We continued to receive periodic complaints about one guest. But, we received no other complaints. Recently, we re-used some more of our existing hardware and implemented another host for our developer workstations. Even though our performance data with Veeam failed to show any week spots, we once again began to receive complaints of this mysterious disconnect issue. We quickly were able to determine that the complaints were about guest vm’s on the most recently implemented host.

Diagnosing the problem was quite a chore. After combing through the host logs using vCenter, we found a few things which, at first, we attributed to flaky firmware on the HP DL360 G5 or perhaps just flaky hardware.

However, one VMware Communities post was frequently returned on Google when using search strings pulled from our logs:

http://communities.vmware.com/message/1357071;jsessionid=7260A1B75C79C9A64D75AD043B0E2C79

At first glance, our thought was, ”how in the world could CD/DVD Rom firmware cause disconnects?” It didn’t make sense. Additionally, the post was about Dell hardware; our hardware is HP. I was in the midst of trying to figure out some of the SCSI error codes I was seeing in the VMKernel log file. I then found the following article:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=289902

After discussing this article with our senior engineer, we thought we’d take a closer look and compare current host against the original host which was having disconnect issues with remote desktop. After reviewing the log files again, we found a line in the error log which raised our eyebrows.

Nov 5 15:37:06 sx4vm001 vmkernel: 0:08:11:01.657 cpu3:4196)WARNING: ScsiPath: 4969: Set retry timeout for failed TaskMgmt abort for CmdSN 0x0, status Failure, path vmhba1:C0:T0:L0

My senior engineer was the person who recently built that host and knew that since the data stores on this host are all iSCSI; hba1 was some other hardware. We looked closer in vCenter and voila. Hba1 was an HP TEAC DVD-ROM drive.

So, we look on the host where the original complains came from during the initial implementation to see if the same condition was true. Voila, hba1 was an HP TEAC DVD-ROM drive. So, we took a leap of faith and made the connection between the last article at the TEAC firmware issue.

As a test, we left two guests alone. On two other guests, we altered the CD/DVD media settings. We disconnected the guest media and changed the device type to client connected. All evening and into the next day, we had no additional complaints.

I then decided to do some quick PowerShell scripting. I already had been writing a script for another VMware task. (I will elaborate on that script in a future blog post).

The beauty of PowerShell is you can quickly make your code re-usable. All I had to do was make two small modifications to an existing script and I was able to change the properties of ALL of my guests in vCenter. I did a bit of housework on it and what appears below is the final script.

You’ll notice I’ve included a function in this small script. This is because it’s a common function from a function library I dot souce in my PowerShell profile.ps1.

So, this is the simple script I quickly through together:

function Test-VIMSnapin { 
if ((Get-PSSnapin -Name VMware.VimAutomation.Core -ErrorAction Stop) -eq $null){ 
Add-PSSnapin VMware.VimAutomation.Core 
} 
} 
Test-VIMSnapin 
$VCServer = Connect-VIServer "vcenter.company.local"</pre>
$VirtualMachines = Get-VM

foreach ($vm in $VirtualMachines) {
$vm | Get-CDDrive | Set-CDDrive -Connected:$false -NoMedia -Confirm:$false | Out-Null
start-sleep -milliseconds 500
}

# Disconnected from vCenter
Disconnect-VIServer $VCServer -Confirm:$False

trap{
Write-Host "ERROR: script execution was terminated.`n" $_.Exception.Message
Disconnect-VIServer $VCServer -Confirm:$False
break
}