Data Center Migration Stories
Planning a data center move? For possibly the “most stressful event in an
IT person’s career,” it is good to have feedback, best practices, and lessons
learned from peers who have been there….and then THERE…..and done that.
For some great responses from the NOREX community to a member planning an
upcoming data center move…
Data Center Migration Stories
Planning a data center move? For
possibly the “most stressful event in an IT person’s career,” it is good to
have feedback, best practices, and lessons learned from peers who have been
there….and then THERE…..and done that.
Read more for some great responses from the NOREX community to a member planning an upcoming data center move:
NOREX Member Response 1
We migrated into a new data center one year ago. It went perfectly smooth and was completed about six hours ahead of schedule over a weekend. The keys to success were:
1. One person responsible for all aspects of the migration
2. Lots of communication and planning of every detail well ahead of time
3. Solid communication and planning with outside vendors that will need to complete tasks ahead of or during the migration
4. Develop a timed schedule and tabletop it well before hand with all involved
5. List of contact names and numbers for anyone and everything that could go wrong
6. Pre-open support tickets when possible for hardware concerns
7. Take pictures, take pictures, take pictures and oh yeah! Take some pictures!
8. Label connections, label connections, and don't forget to label connections!
9. Document the connections on paper
10. Map on paper where everything is going and post the map on a wall during migration
11. Make sure support documents normally stored on local systems are printed or available on systems that will be usable when the data center is offline! This is an easy gotcha!
12. Double and even triple check everything one to two days ahead of the move
13. Develop the working hours schedule, who will be where and when
14. Develop the testing phase schedule. Who will be needed on-site, remote, etc.
15. MAKE SURE SOMEONE IS IN CHARGE OF DONUTS in the morning, PIZZA AND SOFT DRINKS for lunch, and BURGERS FOR DINNER! :-)
NOREX Member Response 2
We migrated our data center 3 years ago using a combination of our Cloud DRaaS replication partner, splitting some of our HA components, and one long final weekend. A lot of it will depend on size of the data center, size of your team, expertise, tolerance for downtime, and budget (buying new stuff makes it a lot easier!) Our data center and team were small... 5 racks, 4 guys, 3 days. If this is similar to your situation, I'd be happy to talk with you further. We were able to take a weekend offline. If we were much larger or had more time to plan, I would have replicated everything to our DR site and back but this was under developed at the time and we were rushed (3 months to plan, build, and move).
Good project management skills were a must. Delegate tasks but keep tabs on progress, road blocks, and milestones. We found our telecom providers by far the hardest to keep on schedule. Construction crew surprisingly kept on track. If your electricians are not familiar with data centers, be sure to be very specific on what you need or they’ll do a standard commercial job which is insufficient for a data center. Good design all around will pay dividends later so don't skimp on documentation (and labeling!). Many of our critical VMs had individual plans.
Make improvements where you can. It’s not often you can make drastic changes to your infrastructure so it’s a good opportunity to cross off several projects at once. We added several subnets and vlans to clean up years of ad-hoc planning, drastically improved the power and cooling, and added HA to several areas that were single points of failure in the old DC. Of course, the more you change the more risk so plan carefully.
Make sure you have passwords for everything and are comfortable with rebooting all of your equipment ahead of time. You don't want to find you lost the boot password to that old NAS everyone forgot about and you can't change the IP on move day. Write mem on your switch configs because that switch attached to that forgotten NAS probably wasn't saved.
If you're doing this over a weekend, remember mistakes are made when people are tired. There will be times for all hands on deck and times to take shifts. We did most of our physical moving on Friday night, got a good night's rest, and returned Saturday morning for the SAN, vSphere, and network wiring. Having our storage vendor on-hand was a life saver to get everything back up and running quickly. By lunch we had some of our first VM's powering up and had most of the infrastructure online by Saturday evening. Sunday was mostly test with the infrastructure team onsite and the rest of the IT team and a few business application owners verifying remotely.
I agree with the donuts, pizza, and burgers! And pictures!
NOREX Member Response 3
The data center move would be classified as the most stressful event in an IT person's career. Most IT professional will encounter an average of 1 data center move in their career. I was fortunate enough to work at different jobs and have managed 3 data center moves, successfully. The data center move is an event that involves ALL IT staff from every department, from sys admins, network engineers, PC support, software support, service desk.... Now is a really good time to update all your DR documentation. Dry runs and table top exercises will be helpful. Spend the money to setup the new data center and run it in parallel with the old and if possible break the move up into as many phases as you need. Make sure IT staff get plenty of rest, after about 15 hours of work, your brain becomes "mushy" :-). Good luck.
On day of the data center move, set up a WAR room. The WAR room will have a conference call number where all IT staff members involved with the move can call in. The WAR room should be managed by a season IT person who will track all open issues on a white board and prioritize what needs to be address first, second, third... Any staff can glance at the white board and see current statuses of what is opened...
The day after the move when employees are back, the WAR room will become the "critical care" room where IT staff who walk the floors/sites can report in user issues that need to be addressed. Again, prioritize issues as they come in and work on them and update statuses on the white board.
For more information, documentation, and discussion, keyword search the NOREX site, or contact your Information Manager.
Not a member? We’ll be happy to send you further information. Clickto let us know how we can help!