Incorrect graph display when aliasing is applied

I am using Grafana 5.2.3, Graph plug-in 5.0.0 and data connection Graphite 5.0.0. My goal is to visualize some performance data from time series DB. My graph chart looks OK initially but it is becomes garbled when I apply aliasing (I have to use aliases as my metric is complex). It seems to me that it happens when the following conditions are met:

  • Metric includes one-to-many relationship (for example: it provides performance metrics of SAN arrays using management servers and one management server reports on many arrays).

  • Data points related to different lower level objects (for example: SAN arrays) related to the same higher level object (for example: management server) were collected with different time stamps.

Please let me present more specific example using the following metric (I omitted names of irrelevant parts for clarity and not disclosing some internal information) delivering performance metric of Dell EMC VMAX SAN array retrieved from Unisphere management server:
m0.m1.m2.m3.m4.m5.m6.m7.<VMAX array SN>.m9.<Unisphere server>.m11.m12.m13.m14

My raw chart in Graph panel looks like expected:

image

Output from Query Inspector:
{
“xhrStatus”: “complete”,
“request”: {
“method”: “POST”,
“url”: “api/datasources/proxy/52/render”,
“data”: “target=m0.m1.m2.m3.m4.m5.m6.m7.*.m9.mgmtserver.m11.m12.m13.m14&from=-3h&until=now&format=json&maxDataPoints=1184”
},
“response”: [
{
“target”: “m0.m1.m2.m3.m4.m5.m6.m7.xxxxxxxxx490.m9.mgmtserver.m11.m12.m13.m14”,
“datapoints”: [
[
744,
1587534414
],
[
551,
1587535867
],
[
546,
1587537288
],
[
557,
1587538720
],
[
553,
1587540145
],
[
675,
1587541555
],
[
546,
1587543007
]
]
},
{
“target”: “m0.m1.m2.m3.m4.m5.m6.m7.xxxxxxxxx303.m9.mgmtserver.m11.m12.m13.m14”,
“datapoints”: [
[
559,
1587534412
],
[
559,
1587535865
],
[
574,
1587537286
],
[
546,
1587538717
],
[
558,
1587540142
],
[
569,
1587541553
],
[
566,
1587543005
]
]
}
]
}

But legend is not very legible so I applied aliasByNode function to extract VMAX array and Unisphere identifiers:
aliasByNode(m0.m1.m2.m3.m4.m5.m6.m7.<VMAX array SN>.m9.<Unisphere server>.m11.m12.m13.m14,8,10)

Now legend looks good but the chart became garbled:

Output from Query Inspector reflects this change:
{
“xhrStatus”: “complete”,
“request”: {
“method”: “POST”,
“url”: “api/datasources/proxy/52/render”,
“data”: “target=aliasByNode(m0.m1.m2.m3.m4.m5.m6.m7.*.m9.mgmtserver.m11.m12.m13.m14%2C8%2C10)&from=-3h&until=now&format=json&maxDataPoints=1184”
},
“response”: [
{
“target”: “xxxxxxxxx303.mgmtserver”,
“datapoints”: [
[
559,
1587534412
],
[
559,
1587534414
],
[
559,
1587535865
],
[
566.5,
1587535867
],
[
574,
1587537286
],
[
560,
1587537288
],
[
546,
1587538717
],
[
552,
1587538720
],
[
558,
1587540142
],
[
563.5,
1587540145
],
[
569,
1587541553
],
[
567.5,
1587541555
],
[
566,
1587543005
],
[
561,
1587543007
],
[
556,
1587544446
],
[
556,
1587544449
]
]
},
{
“target”: “xxxxxxxxx490.mgmtserver”,
“datapoints”: [
[
744,
1587534414
],
[
647.5,
1587535865
],
[
551,
1587535867
],
[
548.5,
1587537286
],
[
546,
1587537288
],
[
551.5,
1587538717
],
[
557,
1587538720
],
[
555,
1587540142
],
[
553,
1587540145
],
[
614,
1587541553
],
[
675,
1587541555
],
[
610.5,
1587543005
],
[
546,
1587543007
],
[
550.5,
1587544446
],
[
555,
1587544449
]
]
}
]
}

So my chart is either illegible or incorrect. How can I fix it?

I have compared data series displayed on both chart versions (slightly later time window than in original post). It is clear that datapoint series “borrow” time stamps but not values from each other.

xxxxxxxxx303 xxxxxxxxx490
Time RespTime1 RespTime2 Time RespTime1 RespTime2
07:46:02 559.0 07:46:02
07:46:54 559.0 07:46:54 744.0
08:11:05 559.0 559.0 08:11:05 647.5
08:11:07 566.5 08:11:07 551.0 551.0
08:34:46 574.0 574.0 08:34:46 548.5
08:34:48 560.0 08:34:48 546.0 546.0
08:58:37 546.0 546.0 08:58:37 551.5
08:58:40 552.0 08:58:40 557.0 557.0
09:22:22 558.0 558.0 09:22:22 555.0
09:22:02 563.5 09:22:02 553.0 553.0
09:45:53 569.0 569.0 09:45:53 614.0
09:45:05 567.5 09:45:05 675.0 675.0
10:10:05 566.0 566.0 10:10:05 610.5
10:10:07 561.0 10:10:07 546.0 546.0
10:34:06 556.0 556.0 10:34:06 550.5
10:34:09 556.0 10:34:09 555.0 555.0

Note:

  • RespTime1 - values from the former chart (a good one).

  • RespTime2 - values from the latter chart (a garbled one).

Any idea what is wrong?