Tag Archives: SDDC

Virtual SAN 6.2 – Deduplication And Compression Deep Dive

Virtual SAN 6.2 – Deduplication And Compression deep dive

by

Virtual SAN 6.2 introduced several highly anticipated product features and in this blog, we’ll focus on some of the coolest ones: Dedupe & Compression. These features were requested by VMware customers and I am glad that we listened to the customer. When talking about Dedupe and Compression, one first needs to determine why an organization would want to use Dedupe & Compression and what these features actually do. One of the many reasons for using Dedupe and Compression is to lower TCO for customers. Customers benefit from space efficiency as the Virtual SAN cluster will not utilize as much storage as it would if it was not using Dedupe and Compression, hence saving dollars. It is also important to note that Dedupe and Compression are supported on All Flash Virtual SAN configurations only.

What are Dedupe and Compression?

The basics of deduplication can be seen in the figure below. What happens is that blocks of data stay in the cache tier while they are being accessed regularly, but once this trend stops, the deduplication engine checks to see if the block of data that is in the cache tier has already been stored on the capacity tier. Therefore only storing unique chunks of data.

Pic 1

So imagine if a customer has lots of VM’s sharing a datastore and these VM’s keep using the same block of data due to a certain file being written too often. Each time a duplicate copy of data is stored, space is wasted. These blocks of data should only be stored once to ensure data is stored efficiently. The Deduplication and Compression operation happens during the destage from the cache tier to the capacity tier.

In case you are wondering how all these blocks of data are tracked, hashing is used. Hashing is the process of creating a short fixed-length data string from a large block of data. The hash identifies the data chunk and is used in the deduplication process to determine if the chunk has been stored before or not.

Together with Deduplication, Compression is enabled at the cluster level. It will not be enabled using Storage Policy Based Management. The default block size for dedupe will be 4k. For each unique 4k block, compression will only be performed if the output block size will be smaller than the fixed compression block size. The goal is to get this 4k block compressed to a size of 2k as seen below. A compressed block will be allocated and tracked in translation maps.

2

Enabling Dedupe & Compression

To enable Dedupe and Compression is not rocket science by any means. Simply go to the Virtual SAN Cluster and enable it from the Edit Virtual SAN Settings screen. Once dedupe has been enabled, all hosts and disk groups in the cluster will participate in deduplication. In this discussion, dedupe domains will be the same as a disk group; therefore, all redundant copies of data in the disk group will be reduced to a single copy, however redundant copies across disk groups will not be deduped. So the space efficiency is limited to the disk group. This means that all components that are in a disk group will share one single copy if multiple components are using the same block.

Dedupe can be enabled and disabled on a live cluster, however there are some implications to doing this. Turning on dedupe means going through all disk groups in the cluster and evacuating all of the data and reformatting the disk group. After this, Virtual SAN will perform dedupe on the disk groups.

So it’s a rolling upgrade. It’s important to remember that dedup and compression are coupled, therefore, once you enable deduplication you are also enabling compression as seen below.

3

IO Intensity

Dedupe is an IO intensive operation. In a non-dedupe world, data is written from tier 1 to tier 2, however with dedupe, things remain the same for the first part. With this in mind, more operations are inherently performed during destaging. IO will go through an additional dedupe path. This will happen regardless of the data being dedupe friendly or not.

Read – When performing a read, extra reads need to be sent to the capacity SSD in order to find the logical addresses and therefore find the physical capacity (SSD) address

Write – During destage, extra writes are required to the Translation Map and to the Hash Map tables. The translation map and hash map tables are used to reduce overheads. So this needs to be accounted for that this overhead is incurred and that a 4k block size is being used.

4

Dedupe Ratio

When looking in the Summary screen for the Datastore, different capacities and dedupe ratios can be viewed. Logical capacity is a new term. It is the capacity footprint seen if Dedupe and Compression are not turned on. So in the example below, the Physical used is 10G and the dedupe ratio is 3.2. Therefore logical capacity is 32G

5

Summary

In summary Dedupe and Compression are fantastic features that are going to be very useful to customers that have all flash configurations, it will reduce their TCO, and from a technical stand point, it is very simple to implement. Customers do not really need to learn anything new so there is no ramp-up on the technology from a learning perspective.

(Courtesy of VMware vSphere Blog)


VMware Social Media Advocacy

Advertisements

VMware Validated Designs Business Continuity and Disaster Recovery

VMware Validated Designs Business Continuity and Disaster Recovery

VMware Validated Designs Business Continuity and Disaster Recovery

In this video, Ryan Johnson demonstrates the failover of the Software-Defined Data Center management, automation and operations solutions – distributed deployments of vRealize Automation, vRealize Orchestrator and vRealize Operations – between regions in the IT Automation Cloud validated design. Follow Ryan Johnson on Twitter as @tenthirtyam on on our podcast at vmware.com/go/podcast. Learn more at vmware.com/go/vvd or follow updates on Twitter @VMwareSDDC – – – – – – – – – – – – – – – – – -…Read More


VMware Social Media Advocacy

VMware Horizon 7 Has Been Released – Instant Clones, Blast Extreme ++

VMware Horizon 7 Has Been Released – Instant Clones, Blast Extreme ++

VMware Horizon 7 Has Been Released – Instant Clones, Blast Extreme ++

When few weeks back we reported on VMware Horizon 7 we did not know when this product will finally hit the GA state. It’s now! You can download Horizon 7 now. It’s a major release which allows scaling up to 10 Horizon PODs across 4 sites with up to 500 000 desktops. So VMware Horizon 7 Has Been Released – Instant Clones, Blast Extreme and more features are in.


VMware Social Media Advocacy

VMware NSX and Cisco ACI: NSX Now Supported on ACI

VMware NSX and Cisco ACI: NSX Now Supported on ACI

In May of 2015, we did a video around VMware NSX vs. Cisco ACI. As part of that video, we made the prediction that VMware NSX and Cisco ACI would not be an either/or discussion in the future (I also did a webinar on the topic that you can download here). At the time, the common question we were getting from clients was if they should be using NSX or ACI. My opinion was that Cisco ACI quite well complimented the feature sets of VMware NSX and that one could really support the other.

Now let’s fast forward to last month (February 2016) to Cisco Live Berlin where an announcement was made that supported just that idea. In  sessions at the conference, they talked about a number of overlay networks in Cisco ACI and specifically mentioned VMware NSX. So what are these use cases? I’m planning on doing a series of videos to explore the topic further. The next video will discuss heavily utilizing Cisco ACI with an overlay of VMware NSX. After that, we’ll look at the opposite – more heavily leveraging the feature sets of NSX on top of the fabric automation feature sets that exist in ACI.

VMware NSX and Cisco ACI: NSX Now Supported on ACI

Watch on GreenPages’ YouTube channel


VMware Advocacy

Zerto: DRaaS in cloud

This is my second post regarding Zerto, but I’d like to focus on the opportunity that it gives to take advantage of the cloud, in particular, VMware vCloud Director or even AWS. We’ll talk about vCD here.

This is the case when a customer has his own infrastructure on premise, but wants to have a parachute in the cloud. Or the opposite: his production is in the cloud, but he wants to keep his investment using the on premise infrastructure as DR.

First of all I’d like to thanks to Vladan Seget, who did a post about Zerto Lab setup here. I’m adding the 3rd case, recovery to/from cloud.

The architecture in this case is slightly different: there’s one more piece, ZCM (Zerto Cloud Manager) that is upon the ZVMs and vCloud Director (s). The installation is simple as ZVM, a light Windows Server and a package to be installed, very few options to set.

2016-02-24_162930.jpg

We’ll access the GUI using the server’s credential.

2016-02-24_163156.jpg

And this is what we get. I apologize for the black boxes, there will be many of them since it’s a production environment. This page summarizes all our customers, name (ZORG: Zerto Organization), a CRM ID for internal purposes (billing, among other) and then the numner of organization they have in vCloud Director and the number of sites on premise that they connect.

Here we should distinguish: there’s one more solution, called by Zerto In-Cloud. The customer can have all in cloud, an organization for production, and another one, usually in another DC, for replication. And nothing at home. Or a combination of this. The only limit is that a single VM can’t be replicated in 2 places. But the same organization can have vApps replicated in cloud and others on premise, for example.

When we’ll create a new ZORG, these is are info requested:

2016-02-24_163431.jpg

Now, let’s take an existing ZORG to explain what’s inside.

2016-02-24_163923.jpg

As before, the ZORG, the ID and, most important, the path where to upload VMDK to preseed big VMs.

2016-02-24_164012.jpg

on permissions tab, we can choose what the customer can do. And, the credentials to access the ZSSP (Zerto Self Service Portal) that we’ll see later. This portal is needed in In-Cloud cases, and when the customer experiences a disaster in his vCenter, so he can access Zerto panel via browser.

2016-02-24_164312.jpg

The organization (s) belonging to the customer are displayed here, with the cloud site (physical one, everyone with its own vCD), nam eof the organization as appears in vCD, and among other info, note the Max limits: we can limit either in number of VM either in storage GB for every customer.

To connect the infrastructure on premise, we need a connector:

2016-02-24_164412.jpg

ZCC, Zerto Cloud Connector – in this case the customer has 2 vCenter connected to his organizations. The connector is a simple VM, deployed automatically by ZCM, that has 2 NICs: one connected to the customer’s organization network, the other one to the main management network. To better understand the role of the connector, it’s like a proxy: in this case Zerto is multitenant, the customer can manage and see only his own VMs.

The last 2 tabs display the groups of VMs replicated (VPG, as Vladan will cover in his next post), and the errors FOR that customer:

2016-02-24_164610.jpg

2016-02-24_164723.jpg

And, lastly, the ZSSP: the portal for the customer to manage his replication. This is the access, where insert the credential previously seen:

2016-02-24_165035.jpg

after which we land here:

2016-02-24_165312.jpg

edit, delete, peer site (vCenter or vCD), status and RPO.

All you need after a catastrophic event (or even much more less…)

Thank you again Vladan.

EMC and VMware introduce VxRail, a new…

EMC and VMware introduce VxRail, a new hyper-converged appliance

EMC and VMware introduce VxRail, a new…

Advertise here with BSA As most of you know I’ve been involved in Virtual SAN in some shape or form since the very first release. Reason I was very excited about Virtual SAN is because I felt it would provide anyone the ability to develop a hyper-converged offering. Many VMware partners have already done this, and with the VSAN […] ” EMC and VMware introduce VxRail, a new hyper-converged appliance ” originally appeared on Yellow-Bricks.com . Follow me on twitter – @DuncanYB.


VMware Advocacy

V2D updated with new NSX integration

Following my last post about Vmware Valitaded Design, I’m happy to review a Nikhil Kelshikar‘s post in his blog allowing us to build a compliant design including NSX feature.

nsx.png

Well, it was already in the last design references, but this integration includes new features such as:

  • Routing Design
  • Security Policy Design
  • Sizing Guidance

I don’t want to overlap Nikhil’s post, so I’ll just write down my first impressions on this update.

First of all it covers vSphere 6.0, and if you’re planning a new SDDC it’s mandatory.

I think that a section of best practices will help all of them (us) involved in a practical SDDC design, allowing to adapt plans according to that rules. The theory explained in the previous issue is great, but a practical guide is even better.

In few words, we can consider this update as a series of advises, a kind of validation for the theory in terms of routing and security, a strong help for architects and a good read for all of those that want to understand better how NSX works in a real environment, maybe even as a smart help for VCP-NV study.