To list these uploaded files:

To download all zip files from AzureBlob storage that start with foo to the local c:\Test\ directory:

New major changes in this release:

Improved Azure Failure Recovery

If an error occurs when uploading, downloading, or deleting because of a network issue or for some unknown reason, the action will be attempted again in four seconds. If there are ten failures then the action has failed and the user is notified. Some known errors, such as if the file does not exist will cause immediate failure as waiting longer will not solve the problem.

New Command Line Arguments for Recursion, Statistics, and Regular Expressions

There are additional command line arguments and a few have evolved in the latest release.
• /FileName (or /F): Can now be used to locate specific files when uploading, downloading, or deleting.
• /Recurse (or /R): When uploading, deleting, listing, or downloading, give this option to iterate all subdirectories.
• /RegularExpression: Uses the specified regular expression to filter the set of blobs in a container.
• /Statistics (or /S): When downloading a large file (currently defined as greater than 32KB), use Statistics to have an ASCII generated progress bar display the current percentage of file that has been downloaded. This option can only be used in a console, it must not be used in batch.

AzureBlobUtility – Command Line Azure Blob Manipulation originally appeared on HiperLogic on May 11, 2012.

This is a quick PowerCLI script that reports VM Name, datastore, name on disk

You can download PowerCLI at http://www.vmware.com/go/powercli

Report the location on disk of Virtual Machines with PowerCLI originally appeared on HiperLogic on March 29, 2012.

This Azure utility allows you to easily integrate legacy applications into Azure by enabling you to PUT, GET, DELETE, and LIST files in Azure storage. It builds on the original AzureBlobCopy.

You can get the code at

$ hg clone https://bitbucket.org/hiperlogic/azureblobutil

Enjoy!

-Rod

AzureBlobUtility – Get/Put/Delete/List Files in BLOB Storage originally appeared on HiperLogic on March 13, 2012.

vSphere 5 is Out! originally appeared on HiperLogic on August 25, 2011.

While not all the Veeam Backup functionality present in the VMware version will be present for Hyper-V, Veeam is bringing these important backup technologies to the Hyper-V platform:

– Both Backup and Replication in One Product.
- File Level Recovery (FLR). The ability to pull individual files from an image based backup to avoid the need for agents.
– A Hyper-V implementation of Change Block Tracking (CBT) to allow Veeam to only backup only the blocks that have changed in the VM. Veeam had CBT even before VMware had an official API for it, so I assume they are leveraging that investment to bring that to HYPER-V.
- Compression and De-Duplication.
- Support for the Free Hyper-V (System Center Virtual Machine Manager (SCVMM) is not required.)

Veeam Backup & Replication v6 is slated to be released in Q4 2011. Note that the v6 release will have a lot of new features for VMware customers as well, Veeam is supporting multiple hypervisors like many virtualization ISV’s, and I anticipate they will continue to innovate on both hypervisor platforms.

Veeam Backup and Replication – Now with Hyper-V originally appeared on HiperLogic on May 16, 2011.

On reboot the Windows 7 desktop is re-composited on the fly with all the new changes, and preserves all user data, user customizations, user installed applications, and so-called “user pollution” where files are stored all over the c:\ drive.

I prefer this approach because it is clean and simple for desktop administrators to manage. Simply schedule updates during the weekly desktop maintenance window or off hours through the Unidesk scheduler.

Unidesk Deployment Deadline

Make sure that your Unidesk management appliance and cache points all have a consistent and correct system time for the system scheduler to work properly, if you did not do this at installation time.

To do this login to the cache points at https://<cachepoint ip>:8003/ with admin and Unidesk1 ( assuming the default login/password) and at your management appliance at https://<management ip>:8004/ with admin and Unidesk1 (assuming the defaults). Go to Configuration->System Time on the left hand side, then validate that the timezone and current system time are correct.

Unidesk – Simple Maintenance With Scheduled Update Deployment originally appeared on HiperLogic on April 3, 2011.

Step 1. Tweak the gold image

See the Unidesk KB article on this topic for more information.

Step 2. Create a non-persistent desktop in Unidesk

unidesk nonpersistent

Step 3. Create a Pano Pool and Assign the Desktops to It

Pano Non-Persistent

Creating Non-Persistent Desktops with Unidesk and Pano originally appeared on HiperLogic on April 3, 2011.

On Windows, Veeam can do application consistent backups through the Veeam Microsoft Volume Shadow Copy Services (VSS) integration for VSS aware applications. Unfortunately Linux does not have a system like Microsoft Volume Shadow Copy Services (VSS) that allows applications to be made application consistent by backup applications like VMware vDR or Veeam Backup.

A simple workaround is to leverage VMware tools ability to call custom freeze/thaw scripts during a quiesced snapshot to enable customers to do the required application quiescence.

The example below is for MySQL, for other applications you need to know the application specific command to quiesce the application and modify the scripts below appropriately.

Step 1. Make sure VMware Tools are installed in the Linux VM
VMware Tools MUST be installed and should be up to date in your Linux Virtual Machine.
Step 2. Determine how to quiesce the application
To quiesce MySQL we will do a mysqldump to create a transactionally consistent backup of mysql without downtime.

Note the disadvantage of this method is that it takes more space since it creates a copy of the database. Alternatively, If you can stop/start mysql during an off hour backup that is a simple way to quiesce ( no transactions can occur if the database is stopped ) as well. You would stop mysql in the pre-freeze, and start mysql in the post-thaw in that case.

Step 3. Create /usr/sbin/pre-freeze-script
Below are the commands to type at the Linux command shell to create the pre-freeze script which will create a transactionally consistent backup of MYSQL and store the resulting statements in /var/mysqlbackup.todaysdate.sql

The resulting .sql file produced by mysqldump contains a set of SQL INSERT statements that can be used to reload the dumped tables at a later time

cat <<EOF > /usr/sbin/pre-freeze-script
mysqldump --single-transaction --all-databases > /var/mysqlbackup.\`date +"%m%d%y"\`.sql
EOF

Check the above command worked properly by examining the created script.

cat /usr/sbin/pre-freeze-script

The result of the cat command above should look like below

mysqldump --single-transaction --all-databases > /var/mysqlbackup.`date +"%m%d%y"`.sql

Make the script executable

chmod 755 /usr/sbin/pre-freeze-script

Step 4. Create the /usr/sbin/post-thaw-script
In this case we don’t have to call anything in the post-thaw so create an empty script :

  touch /usr/sbin/post-thaw-script
  chmod 755 /usr/sbin/post-thaw-script

Step 5. Enable VMware Tools Quiescence
Now check “Use VMware Tools Quiescence” in Veeam Backup, and Veeam will ask VMware tools to quiesce the VM, and VMware Tools will call /usr/sbin/pre-freeze-script and /usr/sbin/post-thaw-script to create the application consistent snapshot.

After the Veeam backup, login to the VM and validate the backup file with the date of the backup exists:

root@vCentos ~# ls -la /var/mysqlbackup.032411.sql
-rw-r--r-- 1 root root 422120 Mar 24 11:59 /var/mysqlbackup.032411.sql
root@vCentos ~#

You can customize the above procedure for any Linux application that provides a method to quiesce itself.

How to Quiesce Linux Virtual Machines with Veeam Backup – MySQL originally appeared on HiperLogic on March 24, 2011.

Usage:

Powershell to Bulk Add a User to Local Admin on Multiple Computers

The script is below, you must change the string MYDOMAINHERE to whatever your domain is.

function ListAdministrators($Group)
{
  $members= $Group.psbase.invoke("Members") | %{$_.GetType().InvokeMember("Name", 'GetProperty', $null, $_, $null)}
  $members
}

function Ping-Server {
   Param([string]$srv)
   $pingresult = Get-WmiObject Win32_PingStatus -Filter  "Address='$srv'"
   if($pingresult.StatusCode -eq 0) {$true} else {$false}
}

if ($args.Length -ne 2) {
 Write-Host "`tUsage: "
 Write-Host "`t`t.\AddToLocalAdmin.ps1 < group or user > <file of machines>"
 Write-Host "`t`tExample: .\AddToLocalAdmin.ps1 FooBarGroup c:\temp\mymachines.txt"
 return
}

#Your domain, change this
$domain = "MYDOMAINHERE"

#Get the user to add
$username = $args[0]

#File to read computer list from
$strComputers = Get-content $args[1]

foreach ($strComputer in $strComputers)
{ 

  if (Ping-Server($strComputer)) { 

      $computer = [ADSI]("WinNT://" + $strComputer + ",computer")
      $Group = $computer.psbase.children.find("administrators")
      # This will list what’s currently in Administrator Group so you can verify the result
      write-host -foregroundcolor green "====== $strComputer BEFORE ====="
      ListAdministrators $Group
      write-host -foregroundcolor green "====== BEFORE ====="

      # Even though we are adding the AD account
      # It is being added to the local computer and so we will need to use WinNT: provider 

      $Group.Add("WinNT://" + $domain + "/" + $username) 

      write-host -foregroundcolor green "====== $strComputer AFTER ====="
      ListAdministrators $Group
      write-host -foregroundcolor green "====== AFTER ====="

   }
   else
   {
      write-host -foregroundcolor red "$strComputer is not pingable"
   }
}

Powershell Script to Bulk Add Domain User to Local Admin originally appeared on HiperLogic on March 24, 2011.

How Cloud Computing Transforms IT Operations originally appeared on HiperLogic on March 11, 2011.

For example, an application owner will come to the VMware team and say “something is not normal with my application, it is slower than normal, what is going on?”.

The VMware admin now has to determine what “normal” is for the application is, determine if the problem is the application or the infrastructure, and package all of this information up to report back to the application owner.

With vCenter Operations a skilled VMware admin can quickly understand what is abnormal to narrow the scope of the issue, quickly identify what the problem and remediate the issue.

vCenter Operations has a very different UI than you may be accustomed to. In the screenshot below, vCenter Operations is reporting on three metrics: Workload, Health, and Capacity:

1. workload score (0-100) – The amount of resources the objects wants to use compared to the resources the object has.
2. health score (0-100). How normal is the object behaving based on past historical behavior of the object.
3. capacity score (0-100). How much capacity do you have on a object, less than 30 days capacity left would be red.

Note I use the word “object”, and object can be many things like a virtual machine etc.

Another detail view:

vCenter Operations Detail

vCenter Operations is packaged as a virtual appliance and comes in three editions, see below for some the basic differences. For the standard edition pricing starts at $50 per VM (Less than 500 VMs). With standard edition you don’t get access to 3rd party integration, customized dashboards, or AD integration.

Note vCenter Operations still requires the VMware admin to interpret the abnormal metrics and create a plan of action, it does not offer a prescriptive recommendation (yet).

Introducing VMware vCenter Operations originally appeared on HiperLogic on March 8, 2011.

A truly random game would be the best offense, since the computer is programmed to reply with a non-random throw based on what a human would do. I thought this would be a fun way to show how you can use PowerShell to create a simple script to create a randomized set of throws to play:

$hand = "rock", "paper", "scissors"
$rand = New-Object system.random
#Just keep repeating this line to get the next throw
$hand[$rand.next(0,3)]

The result after 41 rounds was a PowerShell Win!

Result of PowerShell vs Rock-Paper-Scissors

Fun with PowerShell: PowerShell vs Rock-Paper-Scissors originally appeared on HiperLogic on March 7, 2011.

For example, Google uses a software stack that has allowed them to withstand 1,600 server failures in a 1,800-unit cluster with uninterrupted service. This same distributed model that assumes failure of the underlying resource is how cloud applications are commonly developed today. VMware with their investment in Spring Source has recognized the importance of supporting the next generation of applications that will not be depend on keeping a single OS instance up and running at all costs. The future is going to revolve around applications built and designed to run on unreliable, cheap, expendable resources.

Cloud Applications Don’t Need Hypervisor High Availability and Fault Tolerance originally appeared on HiperLogic on March 7, 2011.

1. Optimized Drivers: The Linux integration components include support for storage and network controllers specifically optimized for Hyper-V R2.
2. Fastpath Boot Support: Boot devices now take advantage of the storage VSC to provide enhanced performance.
3. Timesync: The clock inside the virtual machine will remain synchronized with the clock on the host.
4. Integrated Shutdown: Virtual machines running Linux can be shut down from either Hyper-V Manager or System Center Virtual Machine Manager, using the “Shut Down” command .
5. Symmetric Multi-Processing (SMP) Support: Up to 4 virtual processors (VP) per virtual machine.
6. Heartbeat: Allows the host to detect whether the guest is running and responsive.
7. Pluggable Time Source: A pluggable clock source module is included to provide a more accurate time source to the guest. ( Not supported on 64-bit RHEL ( and by extension CentOS) see the workaround below.

Step 1. Download the Linux Integration Components for Hyper-V R2

First download the integration components download them from Microsoft. You will have to do this form a Windows Machine as it is .EXE file that must be clicked on to extra the ISO image.

Step 2. Mount the ISO image in the Virtual Machine

Mount the ISO image to your existing CentOS/RHEL Virtual Machine executable like below:

Step 3. Prepare your Virtual Machine to Build the Integration Components

Snapshot your VM and have a backup so you can roll back if something goes wrong, don’t do this on a production VM.

Next, you will want to make sure you are running the latest kernel before installing the IC components. Note each time you upgrade the kernel, you will have to rebuild.

yum update
reboot

The development tools are also needed to compile the IC components

yum install kernel-devel
yum groupinstall "development tools"

Finally, we can build the Linux IC components with these commands:

Step 4. Build the IC Components

cp -pR /media/CDROM /tmp/
cd /tmp/CDROM
make
make install

Step 5. Install timekeeping fix and reboot

yum install adjtimex
shutdown -r now

Step 6. Check the kernel modules reloaded on reboot

Check that the vmbus kernel module loaded on startup with this command:

/etc/init.d/vmbus status

Step 7. Turn off eth0, enable seth0

You can now use the

setup

command to configure the networking for seth0, and then configure the the old legacy “eth0” interface to not start on boot (set “ONBOOT=no” in /etc/sysconfig/network-scripts/ifcfg-eth0 using the VI editor).

Finally restart the networking with

/etc/init.d/network restart

Enlighten CentOS or RHEL on Hyper-V R2 originally appeared on HiperLogic on January 30, 2011.

1. Re-configured the RAID from STRIPED to MIRRORED to 1/2 the wiping time.
2. Wiped the disks using the linux shred utility without user intervention.
3. Halted the machine on completion.

This CD is easily mounted across all blades in the chassis via the USB media. A custom solution was developed as some of the popular free wipe utilities were too old to support the latest HP blades.

Here is the guide to creating your own:

Building the SLAX ISO IMAGE

SLAX was chosen because it was simple, had the shred utility installed, and had all the linux drivers needed out of the box. These same principles could be applied to your distro of choice.

Step 1. Download the SLAX iso image, then do the following:

  mount -o loop slax-6.1.2.iso /mnt/loop
  cp -a /mnt/loop /tmp/slax
  cd /tmp/slax

Step 2. Create the following inside the slax directory :

   rootcopy/etc/rc.d/rc.local
   rootcopy/usr/local/sbin/shredit

The contents of this file are like below:
# cat rootcopy/etc/rc.d/rc.local

  #!/bin/sh
   nohup /usr/local/sbin/shredit &
   

#cat rootcopy/usr/local/sbin/shredit

#!/bin/sh 

#delete the existing RAID 0 array. This command is specific to your layout.
echo "Y"  | /opt/compaq/hpacucli/bld/hpacucli ctrl slot=0 ld 1 delete 

#create a RAID 1 array. This command is specific to your layout
/opt/compaq/hpacucli/bld/hpacucli  ctrl slot=0 create type=ld raid=1 

#This says do a one pass overwrite of the device with random data, then follow up with a zero write pass.
shred -n 1 -z -v /dev/cciss/c0d0
#Now halt after completion
halt

Step 3:
Download and install the HP hpacucli RPM. This utility is used to re-create the RAID0 raid to RAID1 to speed up the wipe. SLAX isn’t RPM based, but you can use rpm2cpio on another box to get a CPIO archive to install.
Step 4. Create the ISO
Now just run the

  mkiso_fs.sh

command included with SLAX to build your custom ISO image. Assuming you called it shred.iso, follow the instructions below to mount it on the blades.

Automatically mounting the ISO on the blades

There are a couple ways to mount the ISO image, if you have the ILO Advanced Pack you can use the following method:

Use RIBCL in the OA to download and mount the ISO as a virtual CD drive for one time nooy

1. Put the ISO image somewhere on a webserver on your network.
2. Create this XML in a text file which we will use in the next step via copy/paste, replacing the myloginhere and mypassword here with your admin credentials for the Onboard Administrator (OA). This will mount the shred.iso via http, and configure the blade for a one time boot into this ISO image.

 <RIBCL VERSION="2.0">
  <LOGIN USER_LOGIN="myloginhere" PASSWORD="mypasswordhere">
  <RIB_INFO MODE="write">
    <INSERT_VIRTUAL_MEDIA DEVICE="CDROM" IMAGE_URL="http://someipaddresshere/shred.iso"/>
    <SET_VM_STATUS DEVICE="CDROM">
      <VM_BOOT_OPTION VALUE="BOOT_ONCE"/>
      <VM_WRITE_PROTECT VALUE="YES" />
    </SET_VM_STATUS>
  </RIB_INFO>
  </LOGIN>
 </RIBCL> 

Now copy the above lines, SSH to the OA in the chassis and then type ONE of the following examples depending if you want to shred ALL blades, one blade, or a range of blades.

Applies to All blades in chassis. | is the termination character

  HPONCFG ALL << |

Applies to Bay 1 only

  HPONCFG 1 << |

Applies to Bays 1 through 8

HPONCFG 1-8 << |

Applies to Bays 1, 2, 9 and 10

HPONCFG 1,2,9,10 << |

Now PASTE the XML we copied in the previous step after this.

Finally just type

|

on a line by itself and hit enter to execute the XML string.

Boot the ISO off of a USB key in the OA

The second option works with standard ILO but you need to do the following:

1. Copy the ISO image to a USB key, you should see the file shred.iso on the USB key. The OA will show all files with .ISO to mount.
2. Insert the ISO image into the OA of the chassis
3. Go to the OA web page, go to Enclosure Settings and then DVD drive
4. Mount the image on the blade(s) like in the screenshot below:

connect iso to a blade through the OA

After you power on the blade, it will begin the wiping proceses. It took about 3 hours for the wipe to complete, with each blade having dual 72GB 15k SAS drives. Once the wipe is complete, the blades will shut down.

Create a HP Blade Disk Wipe Utility originally appeared on HiperLogic on January 8, 2011.

Enterprise Java Applications on VMware – Best Practices Guide [PDF]

Java in Virtual Machines on VMware [PDF].

Check out this blogger on Java and VMware as well who also wrote a whitepaper on the topic.

Tuning Java Performance on vSphere originally appeared on HiperLogic on December 7, 2010.

While the setting varies from vendor to vendor, all you do is enable Intel Virtualization in the ESX BIOS, and vSphere will do what it thinks is best with regards to hardware vs software MMU with no intervention required.

If you want to dive deeper, See page 21 of The vSphere Performance Best Practices Guide. under Configuring ESX for Hardware-Assisted Virtualization.

Especially note that vSphere will use the hardware MMU sometimes, and software MMU sometimes, depending on a mix of factors. See:

http://communities.vmware.com/docs/DOC-9882

for the default settings matrix when the VM is set to Automatic for CPU/MMU Virtualization (the default) . See the best practices guide mentioned above for how to change from Auto if you want to override the VMware recommendation like in the screenshot below:

The only tradeoff for EPT is you give up the memory savings of VMware TPS (Transparent Page Sharing). This VMware KB article sums it up very nicely:

http://kb.vmware.com/kb/1021095

In a nutshell, you give up TPS for EPT because for EPT to be really effective large memory pages are used, but using large memory pages really reduces the odds ( i.e. makes it impossible) to find two pages of identical memory to be shared. Under heavy memory pressure, vSphere will break down the large pages into smaller pages in order for TPS to kick in before it has to resort to swapping.

Incidentally Large Memory Pages is one reason Microsoft decided against implementing TPS in Hyper-V. . Microsoft has enabled large pages by default in 2008/2008 R2/Windows 7. While it may be in the future TPS will have diminished importance, it is a nice feature to get ( for free ) today on vSphere.

Using Intel Extended Page Tables (EPT) with vSphere originally appeared on HiperLogic on December 7, 2010.

As of this post the Hyper-V R2 Linux Integration Components v2.1 from Microsoft do not support RHEL6.

Then you will see eth0. ( Note you will not receive the benefits of OS enlightenment until Microsoft supports RHEL6 under Hyper-V. )

Installing RHEL6 on Hyper-V R2 originally appeared on HiperLogic on December 6, 2010.

With vSphere 4.1, it is now fully supported to pass through USB to the ESX host, enabling you to plug the license dongle into your ESX host, and attached that to a virtual machine. The USB pass through even works when you vMotion the VM to another ESX host. The vSphere guide has a list of supported USB license key style dongles that have been validated.

However, there is one large caveat with this solution, and that is that if you need to take the ESX host down that is hosting the USB key, for example to do routine maintenance, or during an unplanned HA event, the VM will no longer be able to access the USB key.

This often means customer prefer to stick with the pre-vSphere 4.1 “tried and true” method of using a USB over IP adapter. This method while costing a couple hundred bucks has the advantage the VM still has access to the USB device at all times, assuming the network stays up.

USB over IP

Virtualizing USB License Dongles – To Pass Through or Not To Pass Through originally appeared on HiperLogic on October 29, 2010.

As a refresher, the VMware .vswp file is used by ESX/ESXi to allow for memory overcommit, each running virtual machine (VM) has a .vswp file whose size is the virtual machine memory MINUS the memory reservation for the VM. The .vswp file is typically deleted when the VM is powered off, and re-created when the VM is powered on. The .vswp files can be configured to be in a centralized datastore, or to be stored alongside the associated VM (the default).

With that background, the issue for this customer was that with a centralized swap datastore, it wasn’t clear to them how to take the filer backing the swap datastore down without incurring downtime of a virtual machine to re-create the .vswp in the new location via a power VM power off/power on operation.

Luckily, the vswp file is also re-created when you do a VMotion to another host, or a Storage vMotion to another LUN. The simple solution was after changing the swap data store location from the current centralized location, to VMotion all VMs off of host A to other hosts, then back. Repeate for each host in the cluster. A quick PowerCLI script took care of the whole process.

When you are done, all .vswp files for running VMs will have been re-created at the new location. You can validate no VMs are using the old swap datastore, then remove it.

Swapping Out Your VMware Swap Datastore With No Downtime originally appeared on HiperLogic on October 18, 2010.