[4102] in BarnOwl Developers
Re: [barnowl/barnowl] Defer failed log messages (#109)
daemon@ATHENA.MIT.EDU (Jason Gross)
Sat Aug 5 22:29:39 2017
Date: Sun, 06 Aug 2017 02:29:36 +0000 (UTC)
From: Jason Gross <notifications@github.com>
Reply-To: barnowl/barnowl <reply+004448c95dd564381614ff6b08cf1610f013a4d84b0a214a92cf00000001159e411092a169ce009427d1@reply.github.com>
To: barnowl/barnowl <barnowl@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
In-Reply-To: <barnowl/barnowl/pull/109@github.com>
----==_mimepart_59867f1060dcb_282de3fc3115e3c38885ec
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
JasonGross commented on this pull request.
> +
+use BarnOwl::Timer;
+use Exporter;
+
+our @EXPORT_OK = qw();
+
+our %EXPORT_TAGS = (all => [@EXPORT_OK]);
+
+$BarnOwl::Hooks::startup->add("BarnOwl::DeferredLogging::_register_variables");
+
+sub _register_variables {
+ my $flush_logs_interval = -1;
+ my $flush_logs_timer;
+ BarnOwl::new_variable_full('flush-logs-interval',
+ {
+ default => -1,
I've set the default to 1 hour.
Relevant discussion:
```
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:26:27 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
My deferred-logging branch currently activated on EPERM and EACCES
(presumably because your tickets died). I'm tempted to add ETIMEDOUT
for AFS blips. What do others think?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:27:42 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
"activated" means?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:31:04 2013 Host: sirius.fac.cs.cmu.edu
From: JHutz <jhutz@CS.CMU.EDU>
A typo for "activates" ?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:31:46 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
I don't have the context necessary to know what it means for "deferred
logging" to "activate". Why would it not always be active, and flush when it
can?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:40:27 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
Erm, sorry. The problem with having it be always active is that
there's no way (AFAIK) to be notified when you get access to a
file/directory, or when writing to it won't cause a connection time
out. So either I have to try to auto-flush logs every
$amount-of-time, or every time a new message comes in. When I was
first writing it, some people didn't like the idea of auto-flushing
every time a new message comes in, or every $n minutes. I don't
recall the exact reasons.
Currently, there are two modes: normal operation, and deferred
logging. In both modes, it adds incoming messages to a queue. In
normal operation, every time the logging thread is activated by the
mainloop (or whatever the thread manager is called), it tries to write
all messages in the queue to the disk, and enters deferred logging
mode if any fail. In deferred logging mode, the logging thread
doesn't do anything. There is a command :flush-logs which will change
the mode to normal operation, and attempt to write the queue to disk
(there's an option about whether or not to drop the ones that fail).
(Note that you can have it auto-flush every so often by adding a timer
that calls :flush-logs, or adding :flush-logs to the recieve message
hook, or something.)
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:41:18 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
This is https://github.com/barnowl/barnowl/pull/109
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:42:25 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
s/Currently/In the current implementation of my pull request/
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:45:02 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
Are there reasons to explicitly qualify the errnos that switch to deferring
logging instead of explicitly qualifying the errnos that are completely
fatal/should be immediately retried?
I guess that's kinda pedantic.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:00:21 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
The other errnos just result in logs getting dropped. I don't recall
the reasoning; I'm happy to switch it to deferring logging on all
errnos. However, if we keep the current behavior, the reason to
specify the errnos like this is that it's nicer to specify two errnos
than a few hundred.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:03:08 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
I think I was more asking what set of errnos is more concise...
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:27:14 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
I'm not entirely sure what the list of errnos set by fopen is.
http://linux.die.net/man/3/fopen points me to malloc and to
http://linux.die.net/man/2/open, which has a list of 22 errnos, and
that list does not include ETIMEDOUT. There are some on that list
that we obviously don't care to retry (e.g., ENAMETOOLONG); most of
them seem not obviously important to include*, but we might want to
just be permissive here.
*The reason I write this patch was so that if I forgot to renew my
tickets, I wouldn't loose all the logs between when I lost tickets and
when I get them back. It also seems nice to not loose logs on AFS
blips. I don't think we really care about the case where you, e.g.,
have too many symbolic links in the file name and want to change it
and not loose the few logs you got before you realized.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:27:31 2013 Host: MAPLE-SYRUP.MIT.EDU
From: Adam Glasgall <glasgall>
AFS adds some extra errno values.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:32:38 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
Which is why I asked if the list of "just give up" errnos is more concise...
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:35:50 2013 Host: sirius.fac.cs.cmu.edu
From: JHutz <jhutz@CS.CMU.EDU>
No. There are a few situations in which you know it might be helpful to
retry later. Everything else should just fail.
The question is not which list is shorter. It's which list contains
the errors you haven't thought of.
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/barnowl/barnowl/pull/109#discussion_r131533212
----==_mimepart_59867f1060dcb_282de3fc3115e3c38885ec
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
<p><b>@JasonGross</b> commented on this pull request.</p>
<hr>
<p>In <a href="https://github.com/barnowl/barnowl/pull/109#discussion_r131533212">perl/lib/BarnOwl/DeferredLogging.pm</a>:</p>
<pre style='color:#555'>> +
+use BarnOwl::Timer;
+use Exporter;
+
+our @EXPORT_OK = qw();
+
+our %EXPORT_TAGS = (all => [@EXPORT_OK]);
+
+$BarnOwl::Hooks::startup->add("BarnOwl::DeferredLogging::_register_variables");
+
+sub _register_variables {
+ my $flush_logs_interval = -1;
+ my $flush_logs_timer;
+ BarnOwl::new_variable_full('flush-logs-interval',
+ {
+ default => -1,
</pre>
<p>I've set the default to 1 hour.</p>
<p>Relevant discussion:</p>
<pre><code>Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:26:27 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
My deferred-logging branch currently activated on EPERM and EACCES
(presumably because your tickets died). I'm tempted to add ETIMEDOUT
for AFS blips. What do others think?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:27:42 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
"activated" means?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:31:04 2013 Host: sirius.fac.cs.cmu.edu
From: JHutz <jhutz@CS.CMU.EDU>
A typo for "activates" ?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:31:46 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
I don't have the context necessary to know what it means for "deferred
logging" to "activate". Why would it not always be active, and flush when it
can?
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:40:27 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
Erm, sorry. The problem with having it be always active is that
there's no way (AFAIK) to be notified when you get access to a
file/directory, or when writing to it won't cause a connection time
out. So either I have to try to auto-flush logs every
$amount-of-time, or every time a new message comes in. When I was
first writing it, some people didn't like the idea of auto-flushing
every time a new message comes in, or every $n minutes. I don't
recall the exact reasons.
Currently, there are two modes: normal operation, and deferred
logging. In both modes, it adds incoming messages to a queue. In
normal operation, every time the logging thread is activated by the
mainloop (or whatever the thread manager is called), it tries to write
all messages in the queue to the disk, and enters deferred logging
mode if any fail. In deferred logging mode, the logging thread
doesn't do anything. There is a command :flush-logs which will change
the mode to normal operation, and attempt to write the queue to disk
(there's an option about whether or not to drop the ones that fail).
(Note that you can have it auto-flush every so often by adding a timer
that calls :flush-logs, or adding :flush-logs to the recieve message
hook, or something.)
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:41:18 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
This is https://github.com/barnowl/barnowl/pull/109
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:42:25 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
s/Currently/In the current implementation of my pull request/
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 14:45:02 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
Are there reasons to explicitly qualify the errnos that switch to deferring
logging instead of explicitly qualifying the errnos that are completely
fatal/should be immediately retried?
I guess that's kinda pedantic.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:00:21 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
The other errnos just result in logs getting dropped. I don't recall
the reasoning; I'm happy to switch it to deferring logging on all
errnos. However, if we keep the current behavior, the reason to
specify the errnos like this is that it's nicer to specify two errnos
than a few hundred.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:03:08 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
I think I was more asking what set of errnos is more concise...
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:27:14 2013 Host: dr-wily.mit.edu
From: Jason S Gross <jgross>
I'm not entirely sure what the list of errnos set by fopen is.
http://linux.die.net/man/3/fopen points me to malloc and to
http://linux.die.net/man/2/open, which has a list of 22 errnos, and
that list does not include ETIMEDOUT. There are some on that list
that we obviously don't care to retry (e.g., ENAMETOOLONG); most of
them seem not obviously important to include*, but we might want to
just be permissive here.
*The reason I write this patch was so that if I forgot to renew my
tickets, I wouldn't loose all the logs between when I lost tickets and
when I get them back. It also seems nice to not loose logs on AFS
blips. I don't think we really care about the case where you, e.g.,
have too many symbolic links in the file name and want to change it
and not loose the few logs you got before you realized.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:27:31 2013 Host: MAPLE-SYRUP.MIT.EDU
From: Adam Glasgall <glasgall>
AFS adds some extra errno values.
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:32:38 2013 Host: MAPLE-SYRUP.MIT.EDU
From: <rnjacobs>
Which is why I asked if the list of "just give up" errnos is more concise...
Class: barnowl Instance: deferred-logging
Time: Thu Jun 27 15:35:50 2013 Host: sirius.fac.cs.cmu.edu
From: JHutz <jhutz@CS.CMU.EDU>
No. There are a few situations in which you know it might be helpful to
retry later. Everything else should just fail.
The question is not which list is shorter. It's which list contains
the errors you haven't thought of.
</code></pre>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/barnowl/barnowl/pull/109#discussion_r131533212">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AERIyR3JaFCC8AiT8cgL3uTO05VFf4z5ks5sVSUQgaJpZM4AWArI">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AERIySEp4O1pNV0VCrbNa3FXGM0NwVvVks5sVSUQgaJpZM4AWArI.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/barnowl/barnowl/pull/109#discussion_r131533212"></link>
<meta itemprop="name" content="View Pull Request"></meta>
</div>
<meta itemprop="description" content="View this Pull Request on GitHub"></meta>
</div>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/barnowl/barnowl","title":"barnowl/barnowl","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/barnowl/barnowl"}},"updates":{"snippets":[{"icon":"PERSON","message":"@JasonGross commented on #109"}],"action":{"name":"View Pull Request","url":"https://github.com/barnowl/barnowl/pull/109#discussion_r131533212"}}}</script>
----==_mimepart_59867f1060dcb_282de3fc3115e3c38885ec--
