RHM Data Center Rebuild

Posted by Dion Todd May 27th, 2015 6,192 Views 0 Comments

For most of this year I have meant to rework our data center. We host everything ourselves in house because we send about 3 million emails a month through 10 virtual servers and our website gets about 300,000 visitors a month itself. Also with the equipment in hands reach, I can write scripts and configure it to do anything that I want immediately without having to call someone. For us, third party hosting has never worked out, not to mention that for what we need, it would be crazy expensive compared to just paying the light bill and Internet.

I have had my own data center and hosted my own websites since the mid 90s and taught others how to as well. I enjoy it. There have been many ups and downs but we have learned a lot of things along the way. We have had servers taken over by Russians, picked up Internet worms, lost RAID systems and struggled along with drives hanging out the side of the servers, but we made it and eventually got better at it. We maintain near 100% uptime when things go our way, that has not been lately.

A friend of ours was recently parting out a web hosting setup and I agreed to help him with it for some of the parts. So I picked up some nice equipment to upgrade the things that we had in the process. It is all used equipment, but newer than we had and we got two nice APC rack mount battery backups in the process. 

Over the last few months our data center has really fallen into disarray due to a domino effect of something breaking and having to bandage it up the best we could to continue on. New(er) servers become stacked on old broken ones, cables get stretched everywhere and at some point you just need to rip it all out and start over with a spring cleaning. Well, that is what we have been doing the last two weeks. I am very tired and seemed to be bruised all over and sore, but I am happy because it was a job well done. My nephew Jarrett, Sylvia and I have completely reworked the RHM Data Center and it is now in much better shape. We did all of the work ourselves.

The server room has 2” thick insulation and plywood over the top of the regular walls to help protect it from the summer heat like a freezer. We picked up a couple of air conditioners designed for server rooms, but they began to trip the circuit breakers. So we had to pull two new power circuits into the room as well. We picked up a used equipment rack off of Craigslist to house the servers that we picked up off of eBay. Now that is all put together, I am happy with the way that it turned out and believe that it will serve us for years to come. 

Now that we have the power circuits and plenty of cooling, I can look into load balancing the website to handle much more traffic, be much faster, and redundant. Also we can set up a development sandbox site to test new things first without breaking www.refreshinghope.org. 

Here are some pictures of the fun that we have had over the last two weeks:

1) Before we began - Emergency after emergency with failing parts had left it a mess with cables and servers everywhere. The small window A/C could barely keep the room below 90 degrees and we had to turn some equipment off to avoid overheating even more. 

2) We cleared the room out and began assembling a proper equipment rack.

3) Teaching my nephew how to reseat the memory as the server threw an error after being bumped around. 

4) Pulling two new runs of wire into the room to handle the extra equipment safely without tripping the breakers.

5) Wiring the new circuits into the breaker box. 

6) Completed server rack with all the equipment in it and a real server room air conditioner. We have two A/Cs now for load balanced cooling. 

7) All done! (Still had to cover the wall backup where we had to cut through.) 

A big thank you to everyone that has supported Refreshing Hope Ministries. It is certainly helping and we could not do this without you. We are also so encouraged by those who send us cards and uplifting letters. Sylvia and I pray over the donations that come in and ask the Lord to return them 100 fold. I pray that you are richly blessed. The new changes will enable us to reach many more people with the Good News of Jesus around the world and continue our mission.