Go Back   Admin Zone Forums > The Resource Zone > Articles

Articles A Collection of Articles, Interviews, and Other Resources.

Reply
 
Thread Tools

  #1  
Old 01-25-2005, 05:23 PM
Ted S Ted S is offline
Tazmanian
 
Real Name: Ted
Join Date: Feb 2004
Admin Experience: Guru
Location: Southern California
Posts: 3,102
Ted S is a jewel in the rough
Default What do you mean you donít have a backup?
Running a forum takes time, energy and money no matter how small the site or what its intentions are. Yet for all the time and energy people put into their sites many never think to protect them from a database crash or system failure. While this may shock some it shouldnít be all that surprising. After all, most sites havenít had a major crash or donít consider themselves big enough to need any backups. At the end of the day most people just donít spend the time dealing with pesky tasks like backups but when things go wrong they wish they had!

Backing up for small sites, why bother?
So your forum has 50 members and 500 posts, itís small, there arenít even people on most of the time. So why would you need a backup, after all, itís not like the data is going to corrupt and if it does, itís not a big deal. For most forums starting is the hardest part and itís even been said that once you get past a certain mark any decline, any lack of activity will mean the instant death of the forum. If your site has members and has posts youíve already started succeeding and to loose all the time and effort, even if it only has amounted to 500 posts is a huge waste. Furthermore, while your post count may not have crossed the million mark yet, your forum is growing and restarting it from zero is going to be nearly impossible.

Backing up for non-profit sites, who cares and who pays?
You have a successful site with a fair number of posts but you donít make much money off it and you may even be loosing money in the long run, your users donít pay you for the forum so why on earth would you put the time or money into a backup. Making money is not the goal of all communities and while no one wants to loose funds, not having cash should never be a reason to let your forum teeter on the edge of destruction. Backups donít require funds or expensive programs, just disk space and if you donít have enough space to make a backup every now and then, you probably are going to out grow your host every, very soon. If your site is trying to become profitable but has yet to reach that point and you donít back it up you are basically running a business without a safety net Ė loosing something you invested hours and hours and possibly thousands of dollars into for the sake of saving a little time here and a little money there is not a wise move on any level.

Backing up for large sites, itís too hard!
You have a big site and by big weíre talking hundreds of thousands of posts, tons of attachments and more users then you can count to but with all that data backups take a ton of hard drive space and require some serious time so you only do them when youíre really bored and sometimes not even then. Itís true, backing up a 2gig database is a pain no matter how you cut it, especially if your server is already being pushed to its limits for resources but thatís all the more reason to do the backup. Large forums are the most susceptible to database corruption and crashes because of the resources they consume and the number of concurrent connections and inserts. There are very few big forums which have not gone down for a day, had a server crash or lost some amount of data at some point yet many still do not think to run backups on a consistent basis because of the effort they think it involves.

The truthÖ you have to backup.
The truth is no matter how big or small your site is you have put time and money into it, users of some count visit your site to be a part of a community and your site is inherently valuable to someone, probably you. Loosing your site because of data corruption is not just a nightmare, it can and does happen all the time. In December of 2004 alone ScubaBoardís databases crashed twice and we lost 2,000 posts, a bunch of new members and a bunch of repeat members from 3 days of downtime. The only lucky part was that it all happened of the holidays when almost no one was online and we had 5 backups sitting there waiting for us. When we finally identified the problem it turned out that our post table had been completely wiped due to some serious corruption. Had the backups not worked we would have lost 850,000 posts in the blink of an eye and it can happen to any forum at any time.

The truthÖ backs take almost no time.
If youíve ever run a manual backup either through shell or even clicking around in your control panel you know that the problem with backups is they take time and more importantly, require you to remember to perform them. For years my version of a backup meant running a few commands in ssh that I had saved to a text file. The system worked and was even used to perform a recovery once but as time went on I began realizing that my backups would often be spaced out by 3 or 4 days or even as much as a week. This meant that had the site crashed I could have lost thousands of posts (when the forum was younger hundreds of posts but the effect would have been just as bad).

The truthÖ automated backups.
When I finally realized how important backups were it became obvious that the only solution was an automated solution. Machines donít forget to perform backups, they donít go out of town and they donít mind waiting. Every forum program out there has a backup solution, internal, addon or external. For vBulletin there is a free backup hack by Brad.loo which you can find at http://www.vbulletin.org/forum/showthread.php?t=62149. This hack isnít fancy but it uses vBulletinís internal cron system to perform the backup without the need for shell access or any advanced systems. For phpBB TerraFrost has written a similar hack which can be found at http://www.phpbbhacks.com/download/2170/. Most other forums have their own backup programs or hacks that you can find at their site.

The alternative solution to these addons is to use a completely external cron script to go in, backup the database and perform any related tasks. For larger sites or sites on small hosting accounts a major factor in backups is space and to save on space I asked my system admin to come up with a simple backup solution that should work for just about any forum. This script (source below) logs in to any sql database and backs it up to a compressed gz file using the gzip program found on nearly all linux machines. Beyond simply creating backups this script also goes out and prunes old backups leaving only as many as you want (I like to keep 5).
The code:
Code:
#!/usr/bin/perl

$BACKUPPATH="/usr/backup";	#path do backup directory
$BACKUPPREFIX="backup";		#name to begin our backups with
$DBNAME="FORUM DATABASE";		#name of database
$DBUSER="ROOT";		#user with access to database
$DBPASS="PASSWORD";		#path to db
$MYSQLDUMP = "/usr/local/bin/mysqldump"; # path to mysql dump binary program
$LSOPTS="-tr";			#options to sort in reverse by creation date

$KEEP=5;			#number of backups to keep

#$DOM=`date +%d%b%S`;		#replace this line with the line below it
				#to get rid of the seconds at the end of the
				#filename (used for testing)
$DOM=`date +%d%b%S`;		


#---------------------shouldn't have to change anything below here

chomp $DOM;

$FILENAME = "$BACKUPPREFIX-$DBNAME-$DOM.sql";

#========== deal with local backups ==========#
#get existing backups
@backups=`ls $LSOPTS $BACKUPPATH/$BACKUPPREFIX-$DBNAME-*`;

#count their number
$count = $#backups;
$count++;

#over desired number, kill the oldest
if ($count >= $KEEP){
	chomp @backups[0];
	system "rm @backups[0]";
}

#housekeeping out of the way, let's do a backup
system "$MYSQLDUMP --databases $DBNAMEs --add-locks --extended-insert -u $DBUSER -p$DBPASS | gzip > $BACKUPPATH/$FILENAME.gz";
Copy this code to your favorite text editor (ultraedit, textpad, etcÖ), configure the variables up top with your backup path, database login settings, mysql dump folder and file name and save it as backup.pl. Upload backup.pl anywhere on your server and create a new file named cron.txt with the contents:
Code:
0 2 * * * perl /path/to/backup.pl
Use the shell command ďcrontab cron.txtĒ to load the file and the cron will be set.
This will run the backup script every morning at 2am, if you want to generate a more advanced cron or change the options see http://www.htmlbasix.com/crontab.shtml

Going the extra distance with off site backups.
In an ideal world you do not only backup to your local box but also to an external tape drive, secondary server or home machine. In any event having backups in two places is always better then having them in just one, especially if that one is your main server. After all, if the server crashes because of a bad hard drive with all the backups on it, you may as well have no backups. Luckily many server providers (like ThePlanet.com) and hosts now provide tape or NAS backups for free or a low fee. If your host provides these services take use of them, in many cases the support staff will set it all up for you and my script can easily interface with any mounted drive to make a second set of backups, just ask me for help.

The alternative, doing nothing.
Perhaps you still think backups arenít something you need to deal with. Maybe you have the teenager mindset and believe your forum is invincible, perhaps you donít have the disk space for backups or maybe you just donít care. No matter what the reason if you decide not to backup data you may as well stop working on the site. Sooner or later your data will have corruption and even a little corruption can cause a whole table to crash. If it isnít corruption your server will crash, your host will fail, a hacker will get in or something else will happen. This isnít a pessimistic view, itís a realistic view Ė every site has a problem at some time and unless you donít mind restarting the only way to recovery from that problem is to be proactive and keep backups.

Got questions?
Post them here or feel free to contact me. I will gladly assist anyone with getting backups running on their site. Please note, my script is an example and I take no liability for your backups failing for any reason. When you run your first backup, test it before assuming the system works!
__________________
Ted S
New vBulletin Mod: Helpful Answers Post Rating System
Internet Marketing & Social Media Consultant
Reply With Quote
  #2  
Old 02-04-2005, 07:08 AM
AzhriaLilu's Avatar
AzhriaLilu AzhriaLilu is offline
Tazmanian
 
Join Date: Feb 2004
Admin Experience: Guru
Posts: 11,109
AzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond reputeAzhriaLilu has a reputation beyond repute
Default
Excellent article - the amount of people who I've spoken to and they've complained about losing their forum, then look at me blankly when I state "well, didn't you do a backup".... lol
Reply With Quote
  #3  
Old 02-04-2005, 07:32 PM
Ted S Ted S is offline
Tazmanian
 
Real Name: Ted
Join Date: Feb 2004
Admin Experience: Guru
Location: Southern California
Posts: 3,102
Ted S is a jewel in the rough
Default
Yup yup. We "lost" our photo gallery in a botched upgrade. It took a week for photopost to decide that the best fix was to revert (rather than trying to write a script to rebuild the database from the data files). 45 minutes later we were back online but sadly, another site with the same problem is back at zero because they did not have a backup
__________________
Ted S
New vBulletin Mod: Helpful Answers Post Rating System
Internet Marketing & Social Media Consultant
Reply With Quote
  #4  
Old 02-05-2005, 12:18 AM
PacMan's Avatar
PacMan PacMan is offline
Tazmanian
 
Real Name: pacman
Join Date: Apr 2004
Admin Experience: Advanced
Posts: 1,729
PacMan is a glorious beacon of lightPacMan is a glorious beacon of lightPacMan is a glorious beacon of light
Default
Ted, I was going to start a thread on forum backup strategies, but you beat me to it! My VB database is 360 megs with 240 megs being attachments and I rely on my host for backups (!).

I want to move my attachments into the file system in order to make the database backup smaller. I was thinking of backing up the db on a daily basis and doing a complete backup once a week of the db and attachments. I want to get another hosting account from a different host and use it to store my backups. I guess I could write a shell script to FTP the db to the other server?
Does this sound like a viable backup solution?

I would like other admins to share their strategies with taking backups.
Reply With Quote
  #5  
Old 02-05-2005, 05:14 AM
Ted S Ted S is offline
Tazmanian
 
Real Name: Ted
Join Date: Feb 2004
Admin Experience: Guru
Location: Southern California
Posts: 3,102
Ted S is a jewel in the rough
Default
PacMan,
Don't let me stop you! My article is more about why you have to backup then how you should do it (yes I provide a few ways and an example script but thats far from the complete story).

The option you preposed sounds great. A nighly backup of the critical data which is the most important stuff to have and the easiet to loose (databases corrupt more than attachments and users complain more about no posts then they do about a few missing links)! I recommend you automate the process as much as possible to insure it actually gets done. For me this means an automatic backup every night and the big backup weekly. Ontop of that I do some serious mysql repairs every Sunday night but that really turns into every other Sunday or every Sunday I have time which is why automated = better!
__________________
Ted S
New vBulletin Mod: Helpful Answers Post Rating System
Internet Marketing & Social Media Consultant
Reply With Quote
  #6  
Old 02-01-2007, 10:55 PM
Hot_Water Hot_Water is offline
TAZ Regular
 
Real Name: SWJ
Join Date: Jan 2007
Admin Experience: Beginner
Location: Alberta, Canada
Posts: 51
Hot_Water is on a distinguished road
Default What can I add to my home computer to use as backup.
I was thinking I could just add another hard drive to my existing computer and use it only for backups. Is that possible? If so what size hard drive would I need
and do I need to upgrade the rest of my computer to do it?

This is info of my computer:
HP Pavilion
Intel Pentium 4
CPU 2.93 Ghz
504 MB Ram
__________________
What is, is.
Reply With Quote
  #7  
Old 02-04-2007, 07:34 AM
JoshB's Avatar
JoshB JoshB is offline
TalkSTAR
 
Join Date: Mar 2006
Admin Experience: Advanced
Posts: 1,017
JoshB is a jewel in the rough
Default
You just need a small hard drive, for even the largest forums I would say 2GB is the max. size needed for the db and attachments. So you are only going to need around 10GB to keep the last few backups on. Your current system should be able to support this.
Reply With Quote
  #8  
Old 02-04-2007, 08:03 PM
Hot_Water Hot_Water is offline
TAZ Regular
 
Real Name: SWJ
Join Date: Jan 2007
Admin Experience: Beginner
Location: Alberta, Canada
Posts: 51
Hot_Water is on a distinguished road
Default
THANK YOU VERY MUCH! This is the concrete information I have been looking for. Now I can actually go and get this done.
__________________
What is, is.
Reply With Quote
Reply

  Admin Zone Forums > The Resource Zone > Articles





Currently Active Users Viewing this Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
php script to backup database Tdot vBulletin 15 07-23-2006 03:38 PM
vBulletin Automatic Backup WhSox21 Classifieds 8 01-25-2005 06:35 PM
BackUp Boban Community Organization 3 11-03-2004 09:01 AM
Database backup cronjob via CPanel? DChapman Servers and Hosting 18 07-15-2004 01:08 PM
DB backup question phoenixdown21 Servers and Hosting 6 07-05-2004 11:54 AM


 

All times are GMT -4. The time now is 10:31 AM.


Powered by: vBulletin
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Page generated in 0.07821989 seconds with 13 queries
The Admin Zone © copyright 2003-2014 All Rights Reserved. Content published on The Admin Zone requires permission for reprint.