[BackupPC-devel] Feature request(v3, v4). Add non fatal errors notifications to BackupPC_sendEmail.

[Alexander Moisseev]

In some cases nonfatal error may occurs during backup (e.g. xfer errors due to source host filesystem changes). Incomplete or inconsistent backup (even more regular inconsistent backups) requires user attention.

Craig, would you add to BackupPC_sendEmail notification to users if host has recently backed up but nonfatal errors occured during last backup?

Code below is not real patch against BackupPC_sendEmail. Just for illustrate the conception.

if ( time - $last > $Conf{EMailNotifyOldBackupDays} * 24*3600 ) {
my $subj = defined($Conf{EMailNoBackupRecentSubj})
? $Conf{EMailNoBackupRecentSubj}
: $Lang->{EMailNoBackupRecentSubj};
my $mesg = defined($Conf{EMailNoBackupRecentMesg})
? $Conf{EMailNoBackupRecentMesg}
: $Lang->{EMailNoBackupRecentMesg};
my $firstTime = sprintf("%.1f",
(time - $Backups[0]{startTime}) / (24*3600));
my $days = sprintf("%.1f", (time - $last) / (24 * 3600));
sendUserEmail($user, $host, $mesg, $subj, {
firstTime => $firstTime,
days => $days,
userName => user2name($user),
numBackups => $numBackups,
}) if ( !defined($Jobs{$host}) );
next;
}
+ # [-1] in the lines below implies most recent backup for each host.
+ if ( $Backups[-1]{xferErrs}
+ || $Backups[-1]{xferBadShare}
+ || $Backups[-1]{xferBadFile}
+ || $Backups[-1]{tarErrs}
+ ) {
+ #
+ # Here call sendUserEmail() to
+ # send notification to user about errors in last backup
+ #
+ }
if ( $numBadOutlook > 0
&& time - $lastGoodOutlook > $Conf{EMailNotifyOldOutlookDays}
* 24 * 3600 ) {

--
Alexander

[b***@kosowsky.org]

Post by Alexander Moisseev
In some cases nonfatal error may occurs during backup (e.g. xfer errors due to source host filesystem changes). Incomplete or inconsistent backup (even more regular inconsistent backups) requires user attention.
Craig, would you add to BackupPC_sendEmail notification to users if host has recently backed up but nonfatal errors occured during last backup?
Code below is not real patch against BackupPC_sendEmail. Just for illustrate the conception.
if ( time - $last > $Conf{EMailNotifyOldBackupDays} * 24*3600 ) {
my $subj = defined($Conf{EMailNoBackupRecentSubj})
? $Conf{EMailNoBackupRecentSubj}
: $Lang->{EMailNoBackupRecentSubj};
my $mesg = defined($Conf{EMailNoBackupRecentMesg})
? $Conf{EMailNoBackupRecentMesg}
: $Lang->{EMailNoBackupRecentMesg};
my $firstTime = sprintf("%.1f",
(time - $Backups[0]{startTime}) / (24*3600));
my $days = sprintf("%.1f", (time - $last) / (24 * 3600));
sendUserEmail($user, $host, $mesg, $subj, {
firstTime => $firstTime,
days => $days,
userName => user2name($user),
numBackups => $numBackups,
}) if ( !defined($Jobs{$host}) );
next;
}
+ # [-1] in the lines below implies most recent backup for each host.
+ if ( $Backups[-1]{xferErrs}
+ || $Backups[-1]{xferBadShare}
+ || $Backups[-1]{xferBadFile}
+ || $Backups[-1]{tarErrs}
+ ) {
+ #
+ # Here call sendUserEmail() to
+ # send notification to user about errors in last backup
+ #
+ }
if ( $numBadOutlook > 0
&& time - $lastGoodOutlook > $Conf{EMailNotifyOldOutlookDays}
* 24 * 3600 ) {
--
Alexander

I'm not sure I would favor such an addition.
I already get enough emails from BackupPC for various "fatal" errors
given the nature of laptops not being connected to the network
regularly.

Also, many of my backups have regular "non-fatal" errors due to xfer
errors when the source host filesystem changes (e.g., a file gets
deleted during the backup time) or due to various Windows files that
may be locked and blocked.

IMO, the email notification should only be for severe (i.e. 'fatal')
errors where backups are not happening or completing which otherwise
you might never be aware of.

If one wants to look for fine-grained errors within a backup then one
can go to the web interface and/or examine the log files
directly. Alternatively, one could write a tool to parse the xfer log
files to sort and classify the various error types. Perhaps add it to
something like logwatch if you wish.

My point is that this is a very different exercise than simply
notifying that host X hasn't been backed up in Y days...

[Les Mikesell]

Post by b***@kosowsky.org
I'm not sure I would favor such an addition.
I already get enough emails from BackupPC for various "fatal" errors
given the nature of laptops not being connected to the network
regularly.

It would be great as an option that you could specify somehow in the
hosts file to control whether a user gets email about results and at
what verbosity level. Some people even think they want email for
successful runs.

[Alexander Moisseev]

Post by Les Mikesell
It would be great as an option that you could specify somehow in the
hosts file to control whether a user gets email about results and at
what verbosity level. Some people even think they want email for
successful runs.

I agree with Les. Since BackupPC used in different backup scenarios, configurable notifications would be best solution.

There are cases where messages about nonfatal errors just annoying: it is normal that laptops not connected to the network several days or some files appeared or vanished in mail server maildir during backup process.

But if you backing up server running 24x7, having good backup strategy in place, then errors very rarely happens but sometimes need immediate attention. I like BackupPC as "set it forget it" solution, and don't want browse CGI every day just to see what's going on. In this case I want be just warned that something goes wrong (no need for fine-grained error messages).

Also in some cases backup copy of every file and every version of file make sense.

Post by Les Mikesell
I agree, I wouldn't want notification for evry non fatal errors as most
of the time, it's just some files being deleted or renamed during a
backup (and they'll be catched on the next backup).

If some errors occurs particular files never be catched. For example if source host directory path is very long, mangled filename could not be saved on BackupPC v3 file system.

--
Alexander

[Daniel Berteaud]

Post by b***@kosowsky.org

Post by Alexander Moisseev
In some cases nonfatal error may occurs during backup (e.g. xfer errors due to source host filesystem changes). Incomplete or inconsistent backup (even more regular inconsistent backups) requires user attention.
Craig, would you add to BackupPC_sendEmail notification to users if host has recently backed up but nonfatal errors occured during last backup?
Code below is not real patch against BackupPC_sendEmail. Just for illustrate the conception.
if ( time - $last > $Conf{EMailNotifyOldBackupDays} * 24*3600 ) {
my $subj = defined($Conf{EMailNoBackupRecentSubj})
? $Conf{EMailNoBackupRecentSubj}
: $Lang->{EMailNoBackupRecentSubj};
my $mesg = defined($Conf{EMailNoBackupRecentMesg})
? $Conf{EMailNoBackupRecentMesg}
: $Lang->{EMailNoBackupRecentMesg};
my $firstTime = sprintf("%.1f",
(time - $Backups[0]{startTime}) / (24*3600));
my $days = sprintf("%.1f", (time - $last) / (24 * 3600));
sendUserEmail($user, $host, $mesg, $subj, {
firstTime => $firstTime,
days => $days,
userName => user2name($user),
numBackups => $numBackups,
}) if ( !defined($Jobs{$host}) );
next;
}
+ # [-1] in the lines below implies most recent backup for each host.
+ if ( $Backups[-1]{xferErrs}
+ || $Backups[-1]{xferBadShare}
+ || $Backups[-1]{xferBadFile}
+ || $Backups[-1]{tarErrs}
+ ) {
+ #
+ # Here call sendUserEmail() to
+ # send notification to user about errors in last backup
+ #
+ }
if ( $numBadOutlook > 0
&& time - $lastGoodOutlook > $Conf{EMailNotifyOldOutlookDays}
* 24 * 3600 ) {
--
Alexander

I'm not sure I would favor such an addition.
I already get enough emails from BackupPC for various "fatal" errors
given the nature of laptops not being connected to the network
regularly.
Also, many of my backups have regular "non-fatal" errors due to xfer
errors when the source host filesystem changes (e.g., a file gets
deleted during the backup time) or due to various Windows files that
may be locked and blocked.
IMO, the email notification should only be for severe (i.e. 'fatal')
errors where backups are not happening or completing which otherwise
you might never be aware of.
If one wants to look for fine-grained errors within a backup then one
can go to the web interface and/or examine the log files
directly. Alternatively, one could write a tool to parse the xfer log
files to sort and classify the various error types. Perhaps add it to
something like logwatch if you wish.
My point is that this is a very different exercise than simply
notifying that host X hasn't been backed up in Y days...

I agree, I wouldn't want notification for evry non fatal errors as most
of the time, it's just some files being deleted or renamed during a
backup (and they'll be catched on the next backup).

I wrote a script which parse those Xfer errors, so I can be alerted if
the number of non fatal error is above a threshold. I use it with
Zabbix, and it's really helpful (those interested can take a look at
this:
http://gitweb.firewall-services.com/?p=zabbix-agent-addons;a=blob_plain;f=zabbix_scripts/check_backuppc_sudo;hb=HEAD)

Regards, Daniel