📺 Info Siaran: XiaoMi/rdsn/pull/

foreverneverer · 2021-10-26T04:20:59Z

The pr add two main feature:

support the trace point can be reported to monitoring system
support multi-subtracer

New-Config

+ enable_latency_tracer_report = false
+ latency_tracer_counter_name_prefix = trace_latency

+ [task.LPC_WRITE_REPLICATION_LOG_SHARED]
+   enable_trace = true # true will mark the task will be traced latency if open global trace

Log Format Example

	***************[TRACE:[primary.LPC_LATENCY_TRACE]mutation[3.4.3.4646268]]***************
	TRACE:name=replica_2pc.cpp:165:init_prepare                                      , span=                3117, total=                3117, ts=1634899499401766091 
	TRACE:name=prepare_list.cpp:78:prepare                                           , span=                2468, total=                5585, ts=1634899499401768559 
	TRACE:name=mutation_log.cpp:57:append                                            , span=               63398, total=               68983, ts=1634899499401831957 
	TRACE:name=mutation_log.cpp:133:commit_pending_mutations                         , span=                4190, total=               73173, ts=1634899499401836147 
	TRACE:name=mutation_log.cpp:143:operator()_commit_pending_completed              , span=               22229, total=               95402, ts=1634899499401858376 
	TRACE:name=replica_2pc.cpp:544:on_append_log_completed                           , span=                7926, total=              103328, ts=1634899499401866302 
	TRACE:name=replica.cpp:295:execute_mutation                                      , span=             1308213, total=             1411541, ts=1634899499403174515 
	TRACE:name=replication_app_base.cpp:477:apply_mutation                           , span=                1195, total=             1412736, ts=1634899499403175710 
	TRACE:name=replica.cpp:366:execute_mutation_completed                            , span=               74476, total=             1487212, ts=1634899499403250186 
	          [TRACE:[primary.LPC_LATENCY_TRACE]127.0.0.1:34802]          
	 TRACE:name=prepare.replica_2pc.cpp:327:send_prepare_message                      , span=                1330, total=                1330, ts=1634899499401772653 
	 TRACE:name=prepare.replica_2pc.cpp:639:on_prepare_reply_remote_rec_prepare       , span=               43233, total=               44563, ts=1634899499401815886 
	 TRACE:name=prepare.replica_2pc.cpp:640:on_prepare_reply_remote_reply_prepare     , span=               92377, total=              136940, ts=1634899499401908263 
	 TRACE:name=prepare.replica_2pc.cpp:641:on_prepare_reply                          , span=               37115, total=              174055, ts=1634899499401945378 
	          [TRACE:[primary.LPC_LATENCY_TRACE]127.0.0.1:34803]          
	 TRACE:name=prepare.replica_2pc.cpp:327:send_prepare_message                      , span=                2005, total=                2005, ts=1634899499401806364 
	 TRACE:name=prepare.replica_2pc.cpp:639:on_prepare_reply_remote_rec_prepare       , span=             1269964, total=             1271969, ts=1634899499403076328 
	 TRACE:name=prepare.replica_2pc.cpp:640:on_prepare_reply_remote_reply_prepare     , span=               46020, total=             1317989, ts=1634899499403122348 
	 TRACE:name=prepare.replica_2pc.cpp:641:on_prepare_reply                          , span=               50075, total=             1368064, ts=1634899499403172423 
	          [TRACE:[primary.LPC_WRITE_REPLICATION_LOG_SHARED]aio_task]          
	 TRACE:name=commit.native_linux_aio_provider.cpp:137:submit_aio_task              , span=                3904, total=                3904, ts=1634899499401841919 
	 TRACE:name=commit.native_linux_aio_provider.cpp:144:aio_internal                 , span=                7127, total=               11031, ts=1634899499401849046 
	 TRACE:name=commit.native_linux_aio_provider.cpp:158:aio_internal_completed       , span=                5721, total=               16752, ts=1634899499401854767

hycdong · 2021-10-27T03:25:39Z

I think this pull request adds a new feature, it is more suitable to use feat rather than refactor in title.
Besides, I suggest adding more in pull request description about how to configure a sub-tracer.

foreverneverer · 2021-10-27T04:20:33Z

I think this pull request adds a new feature, it is more suitable to use feat rather than refactor in title. Besides, I suggest adding more in pull request description about how to configure a sub-tracer.

sub_tracer is old concept of origin pr and the code has some explain comment, it shoudn't explain again in this pr again. And I update some comment about it

levy5307 · 2021-10-28T10:53:04Z

 #include <dsn/utility/synchronize.h>
 #include <dsn/utility/flags.h>
 #include <dsn/dist/fmt_logging.h>
+#include <dsn/perf_counter/perf_counters.h>


remove this line

levy5307 · 2021-10-28T10:54:49Z

+
+    std::string name() { return _name; }
+
+    std::string description() { return _description; }


const std::string& description() const

levy5307 · 2021-10-28T10:56:08Z

+
+    dsn::task_code _task_code;
+
+    utils::rw_lock_nr _point_lock;


utils::rw_lock_nr _point_lock; // { std::map<int64_t, std::string> _points; // }

levy5307 · 2021-10-28T10:56:16Z

-    std::shared_ptr<latency_tracer> _sub_tracer;
+
+    std::string _parent_point_name;
+    utils::rw_lock_nr _sub_lock;


levy5307 · 2021-10-28T10:57:42Z

    uint64_t ts = dsn_now_ns();
-    utils::auto_write_lock write(_lock);
-    _points[ts] = stage_name;
+    utils::auto_write_lock write(_point_lock);


{ utils::auto_write_lock write(_point_lock); _points.emplace(ts, stage_name); }

_last_time need also lock

levy5307 · 2021-10-29T06:36:47Z


-    traces.append(fmt::format("\t***************[TRACE:{}]***************\n", _name));
-    uint64_t previous_time = _start_time;
+    utils::auto_read_lock point_lock(_point_lock);


Would these two locks produce dead lock?

Yeah, I will fix

levy5307 · 2021-10-29T06:39:31Z

-    if (time_used < _threshold) {
+void latency_tracer::dump_trace_points(/*out*/ std::string &traces)
+{
+    if (!FLAGS_enable_latency_tracer || !_enable_trace || _threshold < 0 || _points.empty()) {


Does this code have thread safet issues?

_points.empty()

Yeah, seem to be not, has updated

foreverneverer added 2 commits October 26, 2021 12:17

init

4785851

init

700336c

foreverneverer marked this pull request as draft October 26, 2021 04:21

init

06770d9

foreverneverer marked this pull request as ready for review October 26, 2021 08:59

init

33658e4

foreverneverer added the type/config-change PR that made modification on configs, which should be noted in release note. label Oct 26, 2021

hycdong reviewed Oct 27, 2021

View reviewed changes

foreverneverer added 2 commits October 27, 2021 11:32

update test

1ff8d9c

update test

4753669

foreverneverer changed the title ~~refactor: support perfcounter and more information trace log~~ feat: support perfcounter and more information trace log Oct 27, 2021

foreverneverer added 3 commits October 27, 2021 12:13

update formant

241d011

update formant

076883a

update formant

15ceb52

foreverneverer added 8 commits October 27, 2021 13:55

update formant

a76ee6d

update formant

7adc157

update formant

66eea7a

update formant

f93b943

update formant

94ec4b0

fix

7100738

fix

e12fe77

fix

420d2a7

levy5307 reviewed Oct 28, 2021

View reviewed changes

fix

7984634

levy5307 reviewed Oct 29, 2021

View reviewed changes

fix

35943c3

levy5307 approved these changes Oct 29, 2021

View reviewed changes

hycdong approved these changes Nov 1, 2021

View reviewed changes

foreverneverer merged commit 667d84d into XiaoMi:master Nov 1, 2021

foreverneverer mentioned this pull request Nov 3, 2021

refactor: use latest latency tool to add point for write process #951

Merged

foreverneverer mentioned this pull request Jul 6, 2022

Release 2.4.0 apache/incubator-pegasus#1032

Closed


		std::string name() { return _name; }

		std::string description() { return _description; }

Conversation

foreverneverer commented Oct 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New-Config

Log Format Example

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hycdong commented Oct 27, 2021

Uh oh!

foreverneverer commented Oct 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foreverneverer Oct 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foreverneverer Oct 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

foreverneverer commented Oct 26, 2021 •

edited

Loading

foreverneverer commented Oct 27, 2021 •

edited

Loading

foreverneverer Oct 29, 2021 •

edited

Loading

foreverneverer Oct 29, 2021 •

edited

Loading