Exchange admins are a bit confused when it comes to DAC mode. Though most of the them know that it has to be used in most scenarios when deploying Exchange 2010 SP1, they are not sure as to what the function of DAC mode is.
DAC stands for Datacentre Activation Coordination and it works using a new protocol named Datacentre Activation Coordination Protocol (DACP). With Exchange 2010 SP1, all kind of DAG deployments can use DAC mode, even one that has just two DAG members. DAC mode was designed to avoid the “split brain syndrome” that will occur in one particular scenario of datacentre switchover. Let me explain the scenario with the help of an example. (The images used are from TechNet video presented by Scott Schnoll). I have two datacentres with two DAG members in each datacentre. The primary datacentre has the file share witness (hub server) and everything works as normal.
My primary datacentre goes down, let’s say due to a power outage. Both my DAG members and the hub server hosting FSW goes offline. Along with that, my WAN connection goes down as well. This will cause my entire DAG to fail, as there are only 2 DAG members online out of the five in the cluster (4 mailbox servers and one HUB server hosting FSW).
In this scenario, I try to restore the DAG in the secondary datacentre by using an alternate witness server (hub server in 2nd datacentre). Both mailbox servers in the 2nd datacentre comes online as there are now 3 members online (out of 5). Everything works as expected and I am happy.
Let’s say that the power in my primary datacentre gets restored and both my mailbox servers and primary witness server comes online. The servers will come online before the network devices (which is how it is usually). Now, my WAN connection is still down and Exchange servers in both datacentres are up and running. My secondary datacentre has already been activated to serve the clients manually. But, since there are three servers online (out of the 5) in the primary datacentre as well (2 mailbox and 1 hub), the mailbox servers will try to activate databases as well (or will it?). This will lead to both DCs hosting same databases, which cannot and should not happen. DAC mode was developed to make sure that this doesn’t happen.
Now, how does DAC mode make sure that the servers in my primary datacentre doesn’t mount databases when the power is restored? When DAC mode is enabled, it changes the way Active Manager works. When a DAG member gets restarted or when the Exchange Replication service is restarted (when DAC mode is enabled), it sets the DACP bit (in MSExchangeRepl.exe) to zero (which means that it can’t mount any database). Active Manager can only mount a database in only two conditions.
- If it can communicate with all other DAG members, in both datacentres (which won’t be possible in our case as the WAN link is down).
- If it can communicate with atleast one DAG member which has DACP bit set to one (which is a DAG member that is serving clients). In our case, these are the servers in the secondary DC. But, as the WAN connection is down, this won’t be possible.
Hence, our mailbox servers in the primary datacentre won’t mount databases, as the DACP bit will be set to zero, while the servers in the secondary datacentre will continue to work.
This is the only scenario for which DAC mode was designed – to make sure that DAG members won’t mount databases when power gets restored in the primary datacentre, while secondary datacentre is “active” and WAN connection is down.
Once the WAN connection is restored, DAG members in primary datacentre can communicate with all other DAG members and see that the databases are already mounted in the secondary DC & they won’t mount any database, which is the normal behaviour.
Hope this clear the confusion around DAC mode.
Thanks for the explanation!! It has been very useful to understand what happened last weekend as we had to shutdown all our Exchange 2010 SP1 environment (2 sites with 3 mailbox server in each site).
Glad to help Miguel.
Hi,
I want to study all the possible scenarios for DAG. Can you Assist i just name the types. i will study it myself, if its easy to share its links i will highly appreciate.
You did a great work its appreciable.
Thanks in Advance.
Hi Syed,
It’s not as easy as listing and learning. Different business requirements lead to different designs of DAG.
I think it is better to start with TechNet.
Thanks
Hi Rajit, Thanks for this article. I have small clarity here. Initially the primary centre having active mailboxes and due to the power failure the secondary mailboxes act on active node on the secondary data centre.
Now the question is, when the power failure happened, DACP set to zero (0) from 1 due to replication failed to another DAG member @ secondary data centre ? If i misunderstood, please clarify…
Very clear explanation. when i read about DAC mode in Technet, its very confusing.
Now i am clear about DAC mode. Thanks very much for the good article !!!
Thanks Ravi.
HatsOff Ranjit!
Nice Article on DAC
Thanks Raghavendra
Rajith,
Thank you so much to make it easy. Keep writing such a nice tech article.
Thanks Amit
Hi Rajith,
Please suggest how can I configure dag in mulitple sites with users hosted in both location.
Site A
1. CAS/HT1
2 MBX1
Site B
1. CAS/HT1
2. MBX2
You add MBX1 and 2 as DAG members. As DAGs can span Ad sites, things are fine
very nice….explained lovely….really cleared my funda of DAc mode….very helpful….gud work bro.
Hi Rajith. Good Explanation however had one doubt. As you said
“Active Manager can only mount a database in only two conditions.”. The second condition
If it can communicate with at least one DAG member which has DACP bit set to one (which is a DAG member that is serving clients)
Now once the WAN link is restored, DAG members in primary site will be able to contact the DAG member which has a DACP bit set to one.
So in that case, Would the DAG member on the primary site mount the database?
Thanks
Vilas
No.. if DAG knows that there is bit 1 enabled for other DAG members then it understands.. Databases are mounted there. So it will not mount database in primary DC and this is how DAG works. Problem only occur when split brain situation happen due to power outage + WAN network and after sometime Power restore but WAN takes time.
Awesome!! Good explanation.
Thanks Ezakial.
One confusion, at one place you said : “Active Manager can only mount a database in only two conditions.”
1.If it can communicate with all other DAG members, in both datacentres (which won’t be possible in our case as the WAN link is down).
2.If it can communicate with atleast one DAG member which has DACP bit set to one (which is a DAG member that is serving clients). In our case, these are the servers in the secondary DC. But, as the WAN connection is down, this won’t be possible.
and at last you said :
Once the WAN connection is restored, DAG members in primary datacentre can communicate with all other DAG members and see that the databases are already mounted in the secondary DC & they won’t mount any database.
Please understand me.
Where are you getting confused Annie?
Great Rajith, far better article then on Microsoft site…very easy and itself explanable article. Thanks a lot.
Thanks Ravinder.
Impressively simple yet technical explaination. Congratulations, thank you!
Thanks Hernan.
good explanation, thanks for posting this. :)
Thanks Kochumon.
Thanks a lot
My pleasure Anant.
Great job! Very nicely explained.
Thanks Vik.
A great article!!! Very clear description. Wherever the pics are from but they fit perfect to this article!!! Will surely add this link in my blog.
Thanks Akash.
Outstanding Sir. I understand it, but I’ve been searching for a way to explain it to the “smarter” people who sit a few IT seats higher. This should bring it down to their level. Thanks a million. :)
Glad to help.
Good Work. Found it really informative.
Thanks :)
Thanks