Commit graph

10 commits

Author SHA1 Message Date
Brian Goff
d932514e7e Revert "use pubsub instead of filenotify to follow json logs"
This reverts commit b1594c59f5e0d1ac898eacde8d91b1ba33c2b626.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-23 21:43:52 -05:00
Brian Goff
201d0bde99 Use pool for pubsub Publish's waitgroups
benchmark             old ns/op      new ns/op      delta
BenchmarkPubSub-8     1036494796     1032443513     -0.39%

benchmark             old allocs     new allocs     delta
BenchmarkPubSub-8     2467           1441           -41.59%

benchmark             old bytes     new bytes     delta
BenchmarkPubSub-8     212216        187792        -11.51%

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-17 14:36:57 -05:00
Shijiang Wei
b8296053e2 optimize pubsub.Publish function
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2016-02-15 19:25:21 +08:00
Shijiang Wei
092f2d6aed use pubsub instead of filenotify to follow json logs
inotify event is trigged immediately there's data written to disk.
But at the time that the inotify event is received, the json line might
not fully saved to disk. If the json decoder tries to decode in such
case, an io.UnexpectedEOF will be trigged.
We used to retry for several times to mitigate the io.UnexpectedEOF error.
But there are still flaky tests caused by the partial log entries.

The daemon knows exactly when there are new log entries emitted. We can
use the pubsub package to notify all the log readers instead of inotify.

Signed-off-by: Shijiang Wei <mountkin@gmail.com>

try to fix broken test. will squash once tests pass

Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2016-02-15 19:25:16 +08:00
David Calavera
a72c316bf4 Add PubSub topics.
A TopicFunc is an interface to let the pubisher decide whether it needs
to send a message to a subscriber or not. It returns true if the
publisher must send the message and false otherwise.

Users of the pubsub package can create a subscriber with a topic
function by calling `pubsub.SubscribeTopic`.

Message delivery has also been modified to use concurrent channels per
subscriber. That way, topic verification and message delivery is not
o(N+M) anymore, based on the number of subscribers and topic verification
complexity.

Using pubsub topics, the API stops controlling the message delivery,
delegating that function to a topic generated with the filtering
provided by the user. The publisher sends every message to the
subscriber if there is no filter, but the api doesn't have to select
messages to return anymore.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-12-02 16:43:49 -05:00
Alexander Morozov
d5b7b9fa88 Don't use time.After if there is no timeout
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-14 09:14:51 -07:00
Alexander Morozov
879a1a6842 Add docstring to pubsub.Publisher
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-14 09:10:14 -07:00
Chun Chen
33cd97fb23 Fix send on closed channel bug
Signed-off-by: Chun Chen <chenchun.feed@gmail.com>
2015-06-12 15:42:34 +08:00
Michael Crosby
a96214b8c6 Remove publisher if no one is listening
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-01-20 20:21:47 -08:00
Michael Crosby
eff8e3294c Add pubsub package to handle robust publisher
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-01-20 20:21:46 -08:00