22 Nov
2010
22 Nov
'10
12:18 a.m.
Dear Friends
Can anyone help me write a script for incremental backup using rsync to create multiple
restore-point-backup that can be easily restored?
I feel a full backup once in 400 days, with hourly incremental, daily incremental and
weekly incremental backup being rotated in a way in which 1 full backup followed by 24
hourly incremental backups, interspersed with 7 daily incremental backups and 5 weekly
incremental backups may be protection enough for my data. What is the you all's expert
opinion? Kindly suggest a better methodology please.
Would it be better if these backups are compressed and stored in tar format? Kindly guide
please.
Regards and thank you all in advance
Kshitiz
22 Nov
22 Nov
9:13 a.m.
2010/11/22 Kshitiz <kshitij_kotak(a)hotmail.com>om>:
I feel a full backup once in 400 days, with hourly
incremental, daily incremental and weekly incremental backup being rotated in a way in
which 1 full backup followed by 24 hourly incremental backups, interspersed with 7 daily
incremental backups and 5 weekly incremental backups may be protection enough for my data.
What is the you all's expert opinion?
I am assuming you meant 40 days, and not 400 days. :-)
Let us say a full backup takes 30 minutes to restore, each weekly
incremental takes 5 minutes to apply and each daily incremental, 1
minute. Let us further say you have an outage 33 days after your
previous full backup. Now, the time for you to restore is:
30 minutes to restore the last full backup
4 x 5 = 20 minutes to apply the 4 weekly incrementals since the last full backup
5 x 1 = 5 minutes to apply the 5 daily incrementals since the last
weekly incremental
Total time taken to restore = 55 minutes ~= 1 hour.
Now, nobody else apart from you can say whether 1 hour of downtime
while you restore from backups is acceptable or not. The right way to
design your backup schedule is to first determine what the acceptable
limit on this time taken to restore (called MTTR, or Mean Time To
Recover - a little bit of statistics enter here to determine this
"mean") is, and then work backwards. You will find that your entire
backup strategy can change with your acceptable MTTR.
A second issue you haven't addressed is that the smallest incremental
you have is hourly. So then what happens if an outage occurs 55
minutes after the last hourly incremental? Are you fine with losing 55
minutes worth of data?
Binand
10:13 a.m.
On Mon, Nov 22, 2010 at 7:43 AM, Binand Sethumadhavan <binand(a)gmail.com> wrote:
2010/11/22 Kshitiz <kshitij_kotak(a)hotmail.com>om>:
Nice analysis.
I feel we can provide better insight if he lets us know what exactly
he is trying to backup. Since he is using rsync, I assume he is
backing up files at the OS level. Now, what application does he use,
and how often to files get generated/modified at the OS level? That
would decide the frequency of his full and incremental backups.
If we are taking about OLTP here, then logic says the data be stored
in a database, not at the OS level. In such a case, the db will take
care of mttr, etc. The 55 minutes of data loss would be taken care of
by the redo logs (if Oracle).
I have never come across a situation where I had to run hourly
incremental backups at the OS level.
Regards,
NMK.
I feel a full backup once in 400 days, with
hourly incremental, daily incremental and weekly incremental backup being rotated in a way
in which 1 full backup followed by 24 hourly incremental backups, interspersed with 7
daily incremental backups and 5 weekly incremental backups may be protection enough for my
data. What is the you all's expert opinion?
I am assuming you meant 40 days, and not 400 days. :-)
Let us say a full backup takes 30 minutes to restore, each weekly
incremental takes 5 minutes to apply and each daily incremental, 1
minute. Let us further say you have an outage 33 days after your
previous full backup. Now, the time for you to restore is:
30 minutes to restore the last full backup
4 x 5 = 20 minutes to apply the 4 weekly incrementals since the last full backup
5 x 1 = 5 minutes to apply the 5 daily incrementals since the last
weekly incremental
Total time taken to restore = 55 minutes ~= 1 hour.
28 Nov
28 Nov
1:43 p.m.
--------------------------------------------------
From: "Nadeem M. Khan" <nadeem.m.khan(a)gmail.com>
Sent: Monday, November 22, 2010 10:13 AM
To: "GNU/Linux Users Group, Mumbai, India" <linuxers(a)mm.ilug-bom.org.in>
Subject: Re: [ILUG-BOM] Incremental Backup using rsync
On Mon, Nov 22, 2010 at 7:43 AM, Binand Sethumadhavan
<binand(a)gmail.com>
wrote:
Thank you Nadeem & Binand for your reply. Will revert to you asap.
Regards Kshitiz
2010/11/22 Kshitiz
<kshitij_kotak(a)hotmail.com>om>:
Nice analysis.
I feel we can provide better insight if he lets us know what exactly
he is trying to backup. Since he is using rsync, I assume he is
backing up files at the OS level. Now, what application does he use,
and how often to files get generated/modified at the OS level? That
would decide the frequency of his full and incremental backups.
If we are taking about OLTP here, then logic says the data be stored
in a database, not at the OS level. In such a case, the db will take
care of mttr, etc. The 55 minutes of data loss would be taken care of
by the redo logs (if Oracle).
I have never come across a situation where I had to run hourly
incremental backups at the OS level.
Regards,
NMK.
--
http://mm.glug-bom.org/mailman/listinfo/linuxers
I feel a full backup once in 400 days, with
hourly incremental, daily
incremental and weekly incremental backup being rotated in a way in
which 1 full backup followed by 24 hourly incremental backups,
interspersed with 7 daily incremental backups and 5 weekly incremental
backups may be protection enough for my data. What is the you all's
expert opinion?
I am assuming you meant 40 days, and not 400 days. :-)
Let us say a full backup takes 30 minutes to restore, each weekly
incremental takes 5 minutes to apply and each daily incremental, 1
minute. Let us further say you have an outage 33 days after your
previous full backup. Now, the time for you to restore is:
30 minutes to restore the last full backup
4 x 5 = 20 minutes to apply the 4 weekly incrementals since the last full
backup
5 x 1 = 5 minutes to apply the 5 daily incrementals since the last
weekly incremental
Total time taken to restore = 55 minutes ~= 1 hour. 
22 Nov
22 Nov
4:33 p.m.
On Mon, 22 Nov 2010, Kshitiz wrote:
Dear Friends
Can anyone help me write a script for incremental backup using rsync to
create multiple restore-point-backup that can be easily restored?
I feel a full backup once in 400 days, with hourly incremental, daily
incremental and weekly incremental backup being rotated in a way in
which 1 full backup followed by 24 hourly incremental backups,
interspersed with 7 daily incremental backups and 5 weekly incremental
backups may be protection enough for my data. What is the you all's
expert opinion? Kindly suggest a better methodology please.
Would it be better if these backups are compressed and stored in tar
format? Kindly guide please.
Regards and thank you all in advance Kshitiz
Why reinvent the wheel ? Look at rsnapshot. Been using it for a couple of
years now.
regards,
Sharukh.
--
Dr. Sharukh K. R. Pavri. Homoeopath, Linuxer.
It's faster horses, Younger women, Older whiskey and More money. --
Tom T. Hall, "The Secret of Life"
28 Nov
28 Nov
1:42 p.m.
--------------------------------------------------
From: "Dr. Sharukh K. R. Pavri." <lists(a)pavri.net>
Sent: Monday, November 22, 2010 4:33 PM
To: <linuxers(a)mm.ilug-bom.org.in>
Subject: Re: [ILUG-BOM] Incremental Backup using rsync
On Mon, 22 Nov 2010, Kshitiz wrote:
Thank you Dr. Will try and configure it and revert in case of any help /
support required.
Regards Kshitiz
Dear Friends
Can anyone help me write a script for incremental backup using rsync to
create multiple restore-point-backup that can be easily restored?
I feel a full backup once in 400 days, with hourly incremental, daily
incremental and weekly incremental backup being rotated in a way in
which 1 full backup followed by 24 hourly incremental backups,
interspersed with 7 daily incremental backups and 5 weekly incremental
backups may be protection enough for my data. What is the you all's
expert opinion? Kindly suggest a better methodology please.
Would it be better if these backups are compressed and stored in tar
format? Kindly guide please.
Regards and thank you all in advance Kshitiz
Why reinvent the wheel ? Look at rsnapshot. Been using it for a couple of
years now.
regards,
Sharukh.
--
Dr. Sharukh K. R. Pavri. Homoeopath, Linuxer.
It's faster horses, Younger women, Older whiskey and More money. --
Tom T. Hall, "The Secret of Life"
--
http://mm.glug-bom.org/mailman/listinfo/linuxers
5:22 p.m.
New subject: [OT] Re: Incremental Backup using rsync
On Monday 22 Nov 2010, Dr. Sharukh K. R. Pavri. wrote:
Dr. Sharukh K. R. Pavri.
Homoeopath, Linuxer.
It's faster horses, Younger women, Older whiskey and More money. --
If that's the homeopathic formula for living a longer and healthier
life, I'm a convert!
-- Raj
--
Raj Mathur raju(a)kandalaya.org http://kandalaya.org/
GPG: 78D4 FC67 367F 40E2 0DD5 0FEF C968 D0EF CC68 D17F
PsyTrance & Chill: http://schizoid.in/ || It is the mind that moves
29 Nov
29 Nov
10:39 p.m.
New subject: [OT] Re: Incremental Backup using rsync
On Sun, 28 Nov 2010, Raj Mathur (????????? ???????????????) wrote:
On Monday 22 Nov 2010, Dr. Sharukh K. R. Pavri.
wrote:
It's a universal formula. No need to convert.
Sharukh.
--
Dr. Sharukh K. R. Pavri. Homoeopath, Linuxer.
Ignorance is when you don't know anything and somebody finds it out.
Dr. Sharukh K. R. Pavri.
Homoeopath, Linuxer.
It's faster horses, Younger women, Older whiskey and More money. --
If that's the homeopathic formula for living a longer and healthier
life, I'm a convert!
-- Raj 
23 Nov
23 Nov
1 p.m.
On Mon, Nov 22, 2010 at 12:18 AM, Kshitiz <kshitij_kotak(a)hotmail.com> wrote:
Dear Friends
Can anyone help me write a script for incremental backup using rsync to
create multiple restore-point-backup that can be easily restored?
I feel a full backup once in 400 days, with hourly incremental, daily
incremental and weekly incremental backup being rotated in a way in which 1
full backup followed by 24 hourly incremental backups, interspersed with 7
daily incremental backups and 5 weekly incremental backups may be protection
enough for my data. What is the you all's expert opinion? Kindly suggest a
better methodology please.
Would it be better if these backups are compressed and stored in tar
format? Kindly guide please.
Try this http://backuppc.sourceforge.net/
Very simple to deploy and use with web based control panel
Regards and thank you all in advance
Kshitiz
HTH
With regards,
--
--Dinesh Shah :-)
Shah Micro System Pvt. Ltd.
+91-98213-11906
+91-9833-TICKET
http://www.shahmicro.com
http://iopt.in
http://crm.iopt.in
Blog: http://dineshah.wordpress.com
4503
days inactive
4511
days old
8 comments
6 participants
participants (6)
-
Binand Sethumadhavan -
Dinesh Shah (દિનેશ શાહ/दिनेश शाह) -
Dr. Sharukh K. R. Pavri. -
Kshitiz -
Nadeem M. Khan -
Raj Mathur (राज माथुर)