The vSphere DSC – Just another perspective

In the last couple of weeks, I have done rounds of meeting with our customers and discussed ways to automate ESXi build and configuration. The most common piece which I found in each of the environment was vSphere auto-deploy. Today, most of our customers deploy ESXi hosts using auto-deploy and post configuration tasks via host profiles. Majority of question or concerns which I got were related to the host profile. My understanding says that customers tend to find host profiles difficult to understand, which is not the case in reality.

Host profiles are excellent. It’s just you need to fine-tune them initially. You rarely get any issue if you have cracked the host profiles successfully. The key here is to set up a reference host and extracting the host profile from it.

Having said that, let me bring you another perspective on doing the post configuration tasks. Today many of you love to do Infrastructure as a Code and believe in a configuration management ecosystem. When you look around all the configuration management tools, you will find that vSphere Desired State Configuration (DSC) is very close to being a complete solution for vSphere configuration management.

vSphere DSC is an open-source module that provides PowerShell DSC resources for VMware. PowerShell DSC resources follow declarative code style, like most configuration management tools, and allow you to document infrastructure configurations. The module has 71 resources that cover most of the configuration aspects of vSphere infrastructure.

We shouldn’t be looking at vSphere DSC in isolation and rather look at complimenting it with vSphere auto-deploy. Think about this, PXE boot ESXi hosts from vSphere auto-deploy and let vSphere DSC do all the post configurations for you, isn’t that cool!

When you extract the host profile, you get all the configurations of an ESXi host, and at times you need to trim down the configurations to ensure that you have control over it. 

vSphere DSC is just the opposite of this approach. You can start with an empty configuration file and keep adding the resource definitions to it as and when required. vSphere DSC configuration gives a complete picture of configurations that you want to Ensure and allows you to quickly replicate the same in other environments.

Just take a look at the below snippet and a demo of my Lab configuration which does range of things on vCenter and ESXi host.

Concluding this, I would say that vSphere DSC just opens up another way of automating the infrastructure builds and config. The project has come a long way now and has done significant improvements in terms of resource coverage.

Stay tuned with the vSphere DSC project and soon you will get new updates from the VMware PowerCLI team.

Learn More about vSphere DSC: https://github.com/vmware/dscr-for-vmware/wiki



Decrypt PSCredential object password and it’s applications

Hello Everyone,

I feel it’s no more a secret that you can decrypt PSCredential Object and read/get the password in plain text. Wait…, I do not know what is PSCredntial objectThis is what you must be thinking. I feel you stumble upon PSCredential object if you do basic PowerShell for system administration.

Get-Credential is the cmdlet that will prompt you for username and password. once you enter your username and password then its basically a PSCredential object for you.

gc

Now, Let’s take a look at the PSCredential Object.

I have stored credentials in a variable $cred which is now a PSCredential Object. When you do Get-member you will come to know more about this PSCredential Object. Look at the below screenshot to understand more.

gc1

When I get $cred in the last command, It does show you a username and password. but if you notice Password than you will come to know that it’s stored as a secure string. This is good because you do not want PowerShell to store the password in plain text.

However, this is sometimes a need to reuse the same credential to authenticate with some other processes in your PowerShell script which requires plain text password as an input. Also, there is a limitation of the PSCredential Object. PSCredential Object will work on cmdlets that know what a PSCredential Object is. In fact, not all the .Net Classes understand what PSCredential Object is. So if you have a cmdlet which is written in .Net class rather than a PowerShell class than you can’t reuse the PSCredential object. In Order to use this, you need to decrypt the password from PSCredential Object and reuse the password to the respective class. Another example is invoking REST APIs, Not all REST APIs understand PSCredential so this means that you need to pass the username and password as a plain text.

Check the below example script, Here I need to invoke a REST method POST which requires username and password to authenticate. I have 2 parameters Pwd(Password) and Name (Username). This specific API does not understand the PSCredential so I need to pass the credential password in plain text.

Now, if I have this script than obviously, it is not secure because whoever has the access to the script will be able to know the credential which you don’t want to do obviously.

So what is the Solution?  Let’s try something.

Can I access the password directly from the PSCredential object

No, You can’t as it’s stored as a secure string. Look at this example.

gc2

  • $cred.Password will not return you the Password as plain text
  • $cred.Password|Convertfrom-SecureString will give you cipher data rather than a password as a plain text.

So what’s the solution. Well, the solution is in the PSCredential object itself. Do $cred|Get-member. 

gc3

PSCredential object has a method called GetNetworkCredential() method. you can use this method to decrypt the password in PSCredential object.

When I invoke this method and do Get-Member, it will show you the properties of the object and you will find a property called Password. use the last command $cred.GetNetworkCredential().Password and it will return the password in plain text. Please refer to the below screenshot.

gc4

So now I have modified the same script as below,

Conclusion: 

Yes, PSCredential stores the password in a secure string but it has a built-in function GetNetworkCredential() to decrypt the same.

Is it safe to use?

I feel No. Once script execution stops or runtime environment close, variables get disposed and you no longer have access to the variable. However, there are ways in which you can obviously exploit this feature with some tweaks in your Powershell script. for example, I wrote this to a text file. So yes, a PowerShell developer can write this line of code to a txt file and exploit a feature that was intended to be there to help you out.

gc5

I am not sure what is the right way to use credentials in PowerShell script. if you know a method which is definitely a secure way than do let me know with your comments here.


Thanks,

 

 

PsTK1: Getting Started with ‘NetApp PowerShell Toolkit’

Welcome back, As promised earlier, I am back with the new blog series. So let’s get started.

Note: From, Hereon I will be using abbreviation PSTK for ‘NetApp PowerShell ToolKit’, as the same has been referred on NetApp documentation as well.

What is Netapp PowerShell ToolKit? 

NetApp PowerShell Tool Kit (PSTK) is a PowerShell Toolkit packaged with 2 PowerShell Modules which are DataOntap and SANTricity. DataOntap module helps in managing NetApp Storage devices which are managed by the ONTAP management system- such as FAS, AFF, and NetApp Cloud, etc. SANTricity module is used to manage E-Series Storage array and EF-Series flash array.  In this blog series, I will be focusing only on the DataOntap PowerShell module. 

I am highlighting some of the specifications of PSTK here,

Platform: Windows only, Requires PowerShell 3.0 or above and .net 4.5 or above
Is it available on PSGallary? No, Not yet. This means that you can not download it from Install-Module cmdlet of PowerShell
PowerShell Core: No, It does not support PowerShell core yet. So you can’t use this on the Linux Platform yet.
# of cmdlets: 2300 or more for DataOntap Module and ~300 for SANTricity Module.

Documentation and Download link

Why should I learn PSTK?

If you are a Storage admin/Engineer then you would discover that working on PowerShell gives you greater flexibility and automation capabilities compare to any other shell environment. If you have already worked with PowerShell then it’s great. You can simply start using the PSTK module. If you haven’t worked with PowerShell then know this, PowerShell is the simplest scripting platform available for us. Invest some time and you will get it. 🙂

  • PowerShell is primarily a tool for administrators like us
  • PSTK is just a PowerShell module, so if you are already working on any other PowerShell module than you almost require zero additional skillsets to start working on PSTK or any other module in that case
  • The same script can help you to orchestrate things related to the different technology stack. For example, the same script can create a LUN with the help of the DataOntap PowerShell module and further creates a datastore in VMware with the help of PowerCLI (PowerShell Module for VMware vSphere)
  • Everything in PowerShell is an object
  • PowerShell’s command discoverability makes it easy to transition from typing commands interactively to creating and running scripts

How to Install?

Download the .msi installer file and click on install. Ensure you are running with PowerShell 3.0 or above version.

Import-Module

If you are running PowerShell 4.0 or above, By default module will be imported the moment you execute any of the commands which are part of that respective module. However, use below cmdlet if you want to import the module into the Powershell session.

importmodule

Get-Command cmdlet

Below cmdlet will list all the commands which are available to use from DataOntap Module.

If you are entirely new to the PowerShell then I would highly recommend you to refer PowerShell documentation to start your learning with Powershell.


Thanks,

 

 

PSProvider and VMware datastore – PowerCLI

Hello everyone,

I am writing this short blog after a long time. While explaining in-out of PowerShell to some of my friends in person, I discussed about PSProviders. Most of the knowledge about PSProvider is information only and as a script writer we dont really bother about how powershell is playing with different resources (Variable/function/Filesystem/Registery Keys etc) which are used in PowerShell Session or a Script.

However as a VMware Admin I do use PSProvider in background alot in order to move the datastore item from,

  1. datastore to datastore
  2. Datastore to local drive (Windows Drive or Shared Drive) or vice versa

In this Post we will learn about Copy-DatastoreItem cmdlet and PSProviders.

What is PSProvider?

In Simple Term, PSProviders are special repository(data stored within Powershell)  to process the data as it recieves during PowerShell execution. This data is presented in their respective PowerShell drives which are known as PSDrive.

For Ex. See the below command output from Get-PSProvider

Get-Provider.PNG

by default, you get above psproviders which are registry, Alias, Environment, Filesystem, Function and variable. You can also see the respective drives associated to its PSProvider. This means that if you are creating any variable it will be stored in variable: , If you are creating a function then it will be stored in Function: 

check the below image, where i am going into respective drive and able to see the variable which i have created. 

CDVariable

In conclusion, whatever variable/function/etc which I create in powershell gets stored in their respective drives.

vimstore PSProvider. 

vimstore is a one of the PSProvider which you get after connecting to VMware vCenter via PowerCLI. Do this, Connect-VIServer vCenter-ip and then run get-PSProvider cmdlet and you will see additional PSProviders are available to you. These providers are something which provides VMware Inventory and datastore resources to the POwerCLI or PowerShell.
POwerCLIProviders

So, After connecting to vCenter via PowerCli you can see additional PSDrives are available to you, provided by 2 additional PSProviders. I can do cd vmstore: and can actually list the available datastore in the datastore inventory (Simillar to how we list the directories and files in a path) or can list the host inventory.

Once you are connected you can follow below commands to create a New-PSDrvive with ‘Vimdatastore’ PSProvider.

DatastoreDS

Now you have DS: drive which is available to you and you can basically navigate through the same way you do it for any other drive.

Use below command to move data from your local drive to the VMware datastore using PowerCLI. Please note that i am already in DS: , If you are in any other drive then give proper path using vimdatastore drive

copydatastore.PNG

Note: This method is quite helpful in case you are trying to move things around from datastore and you can automate the move operation. also this is an alternate to certificate error which you may receive while moving data from Web Client. For ex, Operation failed when I tried to upload the same ISO using web client.

CertError

Use PowerCLI vimdatastore Psprovider and copy-datastoreitem cmdlet to work around this.

 

Thanks

Jatin

 

 

 

 

 

 

 

 

 

Install-Module -Name VMware.Powercli, behind the proxies!

Are you trying to install PowerCLI from your corporate server? If yes, then you might have faced some sort of errors simillar to this-

nuget

Issue: 

Based on my experience this issue happens mainly because your powershell session is not able to talk to powershell gallery through Nuget package providers. This happens because of corporate proxy connection.

Or

Sometime you don’t have required package provider. In that case ensure FIPS compliant encryption is disabled.

For detailed steps please refer below

  1. Ensure you are running with PSVerion 5 or above. run $psversiontable to check the ps version. 
  2. Ensure you have required package providers
    • Open powershell as an administrator and Run this Get-PackageProvider
    • If you see output as below then you are good. check the step 2.
    • MicrosoftTeams-image
    • If you do not see any package provider than there could be a possibility that FIPS is enabled on your system.
      • Disable FIPS
        • open gpedit.msc
        • Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options
        • In the Details pane, double-click System cryptography: Use FIPS-compliant algorithms for encryption, hashing, and signing and Disable it.

Important: if you do not have default package provider as shown above (more specifically PowerShellGet) then you will not be able to use commands such as install-module/ Find-Module /Update-Module /Save-module etc. 

2. Check the PSRepository

  • Ensure that Powershell gallery is register as PSRepository.
  • Run This command
    • Get-PsRepository
    • PSRepo1.PNG
    • If you see above warning then it means that there is no PSRepo exists.
    • Register PSRepository. 
      • Run this to register a PSRepository.
      • if you recieve below error, then your corporate proxy server is not allowing PSRepository to communiate with your system.
      • PSrepoErr
    • bypass connections via a proxy server.
      • You would require proxy server details (ProxyServerName and Port number)
      • Create a powershell profile by following steps, If its not there. check the below snap and follow exactly the samae
        • New-item -itemtype file -Path $Profile
        • Test-Path $profile
        • notepad $profile
        • Profile
      • With this pase below lines of code in your profile, save and close it. Change your proxy server address and port number
      • This will allow communication to PSgallery after you restart your PSSession.

  • Again run get-Psrepository and you have PSGallery available and registered as ps repository

PSRepApp

3. Now you have Packagemanager and PSRepository. 

4. Run Install-module -name VMware.PowerCLI -Force

5. This will require Nuget and as you have allowed PSgallery communication via proxy, It will first install Nuget and then it will install VMware.Powercli. 

Summary:

Coporate systems do have proxy and sometime FIPS compliance enabled. These 2 security standards stops commincation to PSgallery. Disable FIPS if its enabled and not required and then allow communication to PSgallery via proxy server as explained above.

 

-Jatin Purohit

 

Automate VMFS6 Upgrade – Update-VMfsdatastore cmdlet

By now you must be switching onto vSphere 6.5. During this time you would have also figured it out that there is no online upgrade for VMFS5 to VMFS6, which means that there is a little bit of planning would require to upgrade your VMFS datastores to VMFS6 version.

As of now, the most common approach (and I believe the only approach) is to-

  • Create a temporary VMFS5 datastore
  • Migrate VM, files, and folder from source datastore to temporary datastore
  • Unmount source datastore and delete the source datastore
  • Re-create a new VMFS6 datastore with the same LUN
  • Move back all the VMs and its folder to new VMFS6 datastore

A lot of manual pieces of work right! if its a matter of 5-10 datastore then there should not be any problem in upgrading datastores manually. However, almost all enterprise infrastructures work with hundreds or even thousands of datastores and you certainly need an automated way to do the upgrade.

Update-VMfsdatastore 

PowerCLI has introduced this cmdlet with PowerCLI 6.5, and all the steps which I explained to you earlier are done by this cmdlet.

The “Script”

The above script is just a snippet from my actual production script, You can modify this to suit your environment and need.

Key Take Aways

    • There should be an update required from VMware to allow users to define max number of VMs to storage vMotion at a time. This will eventually speed up the upgrade process. As of now update-vmfsdatastore migrates one VM at a time.
  • During my tests in POC, It was observed that upgrade was failing on datastores which had vmkdump files stored on it, skip any datastore which has vmkdump files, You can skip those datastores by doing something like this-
  • After migrating all active or registered VMs from the source datastore, if swap files or delta.vmdk (snapshots of a virtual disk) files are found, the datastore will not be upgraded.

Visit PowerCLI reference to know more about this cmdlet and its usage.

Hope you have enjoyed this post. Would love to hear back your feedback, challenges and use case regarding VMFS6 upgrade.

Thanks,

Setting Up VSCode for PowerShell in 3 easy step

Recently I have switched from PowerShell ISE to VSCode for script development. I was very comfortable with PowerShell ISE except it’s “intellisense” feature which used to hang my system a lot. Switching to VSCode certainly improved my development experience and gave me a better tool for debugging and testing my Powershell scripts.

VSCode is an open-source source code editor which supports multiple programming languages with the help of VSCode extensions.

Installing and setting up VSCode for PowerShell

  1. Install VSCode
    VSCode is available for Windows, Linux and Mac devices. You can install VSCode from https://code.visualstudio.com/Download
  2.  Install PowerShell extension for VSCode
    • Go to Extension
    • Search for ‘Powershell’
    • Select and install ‘Powershell’ extension
    • Reload
  3. Change your default language mode to PowerShell, Once your PowerShell extension is installed you will see that intellisense in VSCode does not provide suggestions like it does for PowerShell ISE, That is because you have not set up your working environment to PowerShell. To change your working environment to PowerShell do follow below steps-
    • In VSCode, hit ctrl + shift + p 
    • Type “Change Language Mode”
    • and, Select Powershell

With the above steps intellisense will work in the current script file but it will not work in a new script file,

  • To make PowerShell as your default language mode do follow below steps.
    • Got to Settings
    • Search for ‘defaultlan’ and you will find below key 
    •  Set this key to ‘Powershell’ and hit ctrl+s,

That’s all you need to do in order to set up VSCode for PowerShell.

Thanks,