VMware Cloud Foundation 4 Series – Part 1

VMware Cloud Foundation 4 presents an important step in offering a hybrid cloud platform supporting native Kubernetes workloads and management alongside your traditional VM-based workloads.

VMware Cloud Foundation

Cloud Foundation has an automated deployment process, known as Bring-up which deploys standardized workload ready private cloud in just matter of hours! By any measure, VMware Cloud Foundation 4 is a massive release that delivers a complete full stack for all capabilities required, more than I can cover in a single blog. Hence presenting a series where we would look at in-depth concepts of VCF 4. 

VCF 4 Architecture

Before we understand the Architecture, check out the software Building blocks of VCF 4.0:

The new version of VCF 4.0 includes vSphere 7.0, VSAN 7.0, NSX-T 3.0, VRA 8.1, vRLCM 8.1 as well as SDDC manager to manage your virtual infrastructure domains. You can find detailed information on Cloud Foundation Bill of Materials (BOM) here. One thing to note here is you cannot upgrade from VCF 3.x to VCF 4.x. VCF 4 has to be deployed as ‘Greenfield’ deployment only, however this functionality is being currently worked upon and we can expect direct upgrade in coming releases.

Workload Domain

Workload Domain

Workload domain is purpose built logical SDDC instance of one or more vsphere cluster with dedicated vcenter server and dedicated or shared NSX-T instance. It also includes dedicated vSAN Ready nodes. It has automated provisioning and can support up to 15 workload domains.

Most of the customers have question about how many vCenter Server licenses are required during deployment. So to answer this question, you only need single vcenter Server License which is entered during initial deployment and that will support all the vcenter instances deployed within the VCF.

Management Domain

Management Domain

It a special purpose domain that is automatically deployed during Initial Deployment (which is also called as Bring-Up process). It requires minimum 4 hosts and vSAN Storage. vSAN is the only principal storage option for Management Domain.

Management domain is designed to host all infrastructure components like SDDC manager, vcenter server and NSX-T instances and NSX edge instances. It also supports 3rd party management apps like Backup server, Active Directory Servers, Domain controllers, etc.

So, now Management Domain in VCF 4 has smaller footprint as it contains smaller number of VMs. This is because PSCs are now Embedded (External PSC’s are not supported) and NSX Managers and Controllers are integrated into one!

Since the PSC’s are embedded, the functionality of SDDC manager towards has also changed. As new vCenters are deployed, SDDC manager configures a replication ring topology for all the embedded PSC’s and SDDC Manager authenticates to the ring rather than to individual PSCs!

Virtual Infrastructure (VI) Workload Domain

VI Workload Domain

VI Workload Domain contains one or more sphere infrastructures designed to run Customer’s applications and has a dedicated vcenter server deployed in the management Domain.

While deploying the VI workload Domain, Admins have an option of deploying a NSX instance or sharing an NSX instance with existing VI workload domain. Admins also have option to choose between vSAN, NFS or FC as their principal storage options unlike the management domain where vSAN is the only principal storage used.

For the first VI workload domain, the workflow deploys a cluster of three NSX Managers in the management domain and configures a virtual IP (VIP) address for the NSX Manager cluster. Subsequent VI workload domains can share an existing NSX Manager cluster or deploy a new one as stated above.

The management domain and deployed VI workload domains are logical units that carve up the compute, network, and storage resources of the Cloud Foundation system. 

VCF 4 Deployment Types

There are two deployment models used based on the size of the environment.

Consolidated Architecture :

Where customer workloads runs in Management Domain, as simple as that! This model has shared vcenter server where customer workloads are deployed into resource pools. This is recommended for small deployments and it uses minimum of 4 servers. Consolidated deployment uses vSAN as a principal storage and they don’t have any option to select any other type of storage.

Consolidated Architecture

Standard Architecture

The standard architecture aligns with industry best practices separating management workloads with Infrastructure workloads. This is recommended for medium to Large Deployments and it required minimum of 7 (recommended 8) servers to deploy.

Management Workloads are dedicated to Infrastructure and Dedicated VI domains for User workloads. You can run max 15 Workload domains including Management Domain. The important point to note here is vCenter Servers run in enhanced Linked-mode.

Standard Architecture

Key Takeaways

  1. Consolidated deployments utilize one workload domain and one vcenter server.
  2. Standard deployments use separate vCenter server for each Domain.
  3. Multiple clusters are supported in Standard and standard architectures both.
  4. Stretched vSAN deployments are supported.
  5. Consolidated deployment uses vSAN for principal storage.
  6. It’s important to note that VCF 4.1 now supports vVols as Principal Storage in VCF Workload Domains to deliver greater flexibility and storage options.
  7. Each Workload domain can consist of multiple clusters and can scale up to VMware documented maximums!
  8. More information on Consolidated Architecture limitations with Cloud Foundation (70622) – https://kb.vmware.com/s/article/70622

vRealize Automation 8.0 – A lot has changed!

Hello Everyone!

It is always good to refresh new products/new features coming in Industry (Although it’s been a year that vRA 8.0 has been released but it’s still new to many!). So, in this blog post, I’ll try to give an insight on new vRealize Automation 8.0 Architecture and will talk about what exactly has changed.

“vRealize Automation automates the delivery of virtual machines, applications, and personalized IT services across different data centers and hybrid cloud environments.”

vRealize Automation 8.0 is a huge change! What was previously used to known as CAS – Cloud Automation services is now vRealize Automation Cloud and it has its own on-premise version which is nothing but vRealize Automation 8.0. One thing to note here is vRA cloud and vRA 8.0 share the same code base and offer the same user experience, the main difference involves how they are delivered!

Difference between 8.0 and previous versions

More information about the product – https://docs.vmware.com/en/vRealize-Automation/index.html?topic=%252Fcom.vmware.vra.programming.doc%252FGUID-75940FA3-1C17-451C-86FF-638E02B7E3DD.html

vRealize Automation 8.0

vRealize Automation 8.0 brings the vRealize Automation Cloud capabilities to the on-premises form factor. This release modernizes the vRA 8 architecture and capability set to enable enhanced agility, efficiency, and governance in the enterprise.

In Simple Terms, vRA 8.0 is an on-premises solution of vRealize Automation Cloud.

This release of vRealize Automation uses a Kubernetes based micro-services architecture. The new release takes a modern approach to delivering hybrid cloud management, extending cloud management to public clouds, delivering applications with DevOps and managing Kubernetes based workloads.

vRealize Automation Components

It contains four core components –

1. VMware Cloud Assembly: Cloud assembly is a cloud-based service that you use to create and deploy machines, applications, and services to multiple clouds like Google Cloud, Native AWS, Azure and VMC on AWS. Cloud assembly provides multiple key features-

  • Multiple cloud accounts.
  • Infrastructure as a code: Supports blueprints with versioning.
  • Self-service Provisioning.
  • Marketplace: Integrates with website called solutionexchange.vmware.com. This has published built-in blueprints that can be accessed through Marketplace.
  • Extensibility: Built-in feature of vRA. You can use XaaS feature for custom queries.
  • Kubernetes Integration: You can deploy Kubernetes cluster through vRA or you can also import existing Kubernetes cluster to vRA.

2. VMware Service Broker: Service Broker aggregates content in native formats from multiple clouds and platforms into a common/single catalog for easy consumption on VMware Cloud.

3. VMware Code Stream: Code stream is a continuous integration and continuous delivery (CICD) software that enables you to deliver software rapidly and reliably, with little overhead.

4. Orchestrator: takes care of 3rd party integrations, custom scripting and supporting lifecycle action through the Event Broker Service.

Since Cloud Assembly, Code Stream, Orchestrator and Service Broker exist in same appliance, logins are passed between applications. Users can swap seamlessly without logging in each time!

vRealize Automation Architecture

vRA Appliance is powered by a Photon OS Base. It includes native Kubernetes installed on the OS to host containerized services. Now, what does that mean? When the vRealize Automation appliance is deployed, at the first boot, docker is installed and kubernetes clusters are configured. Then, Docker images are stored in a private Docker registry on the appliance.

Role of Kubernetes

Those who doesn’t know what HELM is -It is a package manager for Kubernetes. Helm packages, configures, and deploys applications and services onto Kubernetes clusters. Helm takes images that are stored in a private Docker registry on the appliance and deploys Kubernetes services that are running as pods.

vRealize Automation has 16 core services and all the services are deployed and managed as Pods with each having its own web server running on Kubernetes cluster.

vRealize Automation Architecture

There are two more components which also get installed as a part of vRealize Automation On-premises solution.

  • VMware Lifecycle Manager (LCM): It provides a single installation and management platform for various products in vRealize suite. It delivers complete lifecycle and content management capabilities for vRealize Suite products. It helps customers accelerate time to value by automating deployment, upgrades, and configuration, while bringing DevOps principles to the management of vRealize Suite content. It provides a single installation and management platform.

  • VMware Identity Manager (IDM): It is an Identity as a service (IDaas)) solution. It provides application provisioning, conditional access controls, and single sign-on (SSO) for SaaS, web, cloud and native mobile applications.


Namespaces are a way to divide Kubernetes cluster resources between multiple users. All the core vRealize Automation services run as Kubernetes pods within the namespace called “prelude”. You can explore vRA environment using some of the below commands on vRA appliance using SSH –

  1. To list all the pods running –

kubectl get pods -n prelude

  • To get the number of containers in a pod –

kubectl describe pod <pod_name> -n prelude

  • To list all the services running –

kubectl get services -n prelude

  • To list the deployments running –

 kubectl get deployments -n prelude

(A deployment is responsible for keeping a set of pods running.)

Key Takeaways

  • All the core vRA services run as Pods with each having its own Kubernetes cluster.
  • vRealize Automation 8 cannot be installed on your own Kubernetes environment. It comes in the form of an appliance with all the bits and pieces needed to run vRA 8 and this is the only way VMware can support it.
  • Besides the four core vRealize Automation 8 components, Code Assembly, Service Broker, Code Stream and Orchestrator, two supporting services such as vRealize Identity Manager and vRealize Lifecycle Manager are needed to install and run vRealize Automation 8.
  • If you don’t have a LCM and/or IDM instance running, the easy installer will set one up for you. But you can also use existing LCM and IDM configurations for vRealize Automation 8 as well.
  • There is no Windows IaaS server in vRA 8.0.





I hope this article was helpful. Any suggestions/comments are highly appreciated!

vSphere 7.0 : DRS Re-Designed

Although vsphere 7.0 is a major enhancement in itself with lots of new features added like Kubernetes (Project Pacific), vCenter Server Profiles, vsphere Lifecycle Manager(vLCM), Certificate Management, Refactored vMotion etc.

But the one that caught my eye and is completely re-designed after a span of 15 years is DRS (Distributed Resource Scheduler).

How DRS used to work before vsphere 7.0?

DRS was released back in 2006 and since then it wasn’t changed that much. However, there were a couple of enhancements and changes in vSphere 6.7 (new Initial Placement, NVM Support and enhanced resource Pool reservations). The version of DRS till vsphere 6.7 was a Cluster centric Model. In simple words, the resource utilization was always balanced across the Cluster.

DRS till vsphere 6.7 was a Cluster centric Model.

It’s important to know that DRS regularly monitored the cluster’s balance state once every five minutes, by default, and took the necessary actions to fix any imbalance by live migrating the VM onto the new host using vMotion.

In this way, DRS ensured that each virtual machine in the cluster gets the host resources like memory and CPU, that it needs.

What has changed in DRS vsphere 7.0?

VMware shifted their focus from Cluster centric to Workload Centric Model. Meaning whenever VM runs on an ESXi Host, it calculates “VM DRS Score”. It is totally a new concept!

This score verifies if the VM is scoring enough or is it happy enough on that particular ESXi Host. Let’s see what it is!

VM DRS Score

  • VM DRS Score or also called as “VM Happiness” Score can be defined as the execution efficiency of a virtual machine.
  • Values closer to 0% (not happy) indicate severe resource contention while values closer to 100%(happy) indicate mild to no resource contention.
  • The VM DRS score “works” in buckets. These buckets are 0-20%, 20-40%, 40-60%, 60-80% and 80-100%.
  • A lower bucket score doesn’t directly mean that VM is not running properly. It’s the execution efficiency which is low.
  • DRS will try to maximize the execution efficiency of each virtual machine while ensuring fairness in resource allocation to all virtual machines in the cluster.

How VM DRS Score is calculated?

The calculation of VM DRS Score is per-VM or for a single workload on all the hosts within a cluster.

There are several metrics responsible for VM DRS Score –

  • Performance: DRS looks at CPU Ready Time, CPU Cache behavior and Swapped Memory of the VM.
  • Capacity of the ESXi Host: DRS looks at the headroom that an ESXi Host has and see if an application/workload can burst enough on the ESXi Host that it is running on. This parameter is also called as VM Burst Capacity.
  • Migration Cost: The cost of migration of a VM from one ESXi Host to another. So, you won’t be seeing lots of vMotion happening now! (Only if your DRS is set to Fully-Automated).

Most interesting part is VM DRS Score is calculated every min which gives you far more granular approach.

VM DRS Score is calculated every single min compared to older version where DRS monitored the Cluster’s state every 5 mins.

Cluster DRS Score

As you can see in the diagram, there is something called as Cluster DRS Score which is defined as the average DRS Score of all the virtual machines in the cluster.

Scalable shares:

Very Interesting Concept!

Scalable shares are configured on a cluster level and/or resource pool level.

What’s new is that when you set share level to “high”, it will make sure that VM’s in a Resource pool set to High shares really get more resource prioritization over lower share Resource pools.

In earlier DRS versions, it could possibly occur that VM’s in a Resource pool with shares to “Normal” could get the same resources as a High share Resource pool. Higher share value did not guarantee Higher resource entitlement. This issue is fixed with Scalable Shares.

This setting can be found under Cluster Settings > vSphere DRS > Additional Options > Scalable Shares.

Wrap Up:

We just touched the DRS part. We haven’t discussed about the improved vMotion (or Refactored vMotion) or Assignable Hardware which also plays a major part for DRS.

I hope this article was helpful.

Stay Tuned, and follow the Blog!

For more information on vsphere 7.0, please visit –