WEBVTT

1
00:00:02.830 --> 00:00:11.979
MTS - A/V: Hey, everyone, and welcome to the first RSAC session for ICANN 87. This is the Continuous Improvement Program Work Party session.

2
00:00:12.250 --> 00:00:14.320
MTS - A/V: This call is recorded.

3
00:00:14.440 --> 00:00:16.120
MTS - A/V: And,

4
00:00:16.280 --> 00:00:31.889
MTS - A/V: Yeah, the call is recorded, and this is governed by the ICANN Community Participant Code of Conduct, the ICANN Expected Standards of Behavior, and the ICANN Community Anti-Harassment Policies. Please observe the following guidelines to participate in this session. I will also post them in the chat for your reference. This session is for RSAC members.

5
00:00:31.890 --> 00:00:33.670
Jeff Osborn: And they're inviting guests to that.

6
00:00:33.670 --> 00:00:34.780
MTS - A/V: Lead for this meeting.

7
00:00:35.170 --> 00:00:41.059
MTS - A/V: But, you know, observers are open, for ICANN 87, and at the table, they're totally welcome.

8
00:00:41.320 --> 00:00:52.729
MTS - A/V: If you wish to speak, please raise your hand in the Zoom room, or otherwise directed by the chair. When speaking, please state your name for the record, and speak clearly at a moderate pace. I will now hand the floor over to Naveed.

9
00:00:56.900 --> 00:00:59.449
MTS - A/V: Yeah. Hello, everyone. Good afternoon.

10
00:00:59.930 --> 00:01:07.440
MTS - A/V: Thank you, Daniel. So we today's agenda would be to

11
00:01:07.550 --> 00:01:10.760
MTS - A/V: To discuss the working document that,

12
00:01:11.150 --> 00:01:14.580
MTS - A/V: We have that we have as a base point for.

13
00:01:14.640 --> 00:01:22.660
MTS - A/V: For this CIP working party. So in we already have 2 calls.

14
00:01:22.690 --> 00:01:33.840
MTS - A/V: On, on, on this working party. And, in the last working party, we, we, agreed, to…

15
00:01:33.840 --> 00:01:42.919
MTS - A/V: to start deliberating on the document, the base document that we already have, which provides the

16
00:01:42.920 --> 00:02:00.410
MTS - A/V: the base of the principles that are fixed for this process, and plus it provides a number of exemplary criteria, as well as the indicators. And just as a recall, the indicators are what we, look for.

17
00:02:00.440 --> 00:02:14.579
MTS - A/V: to measure at the end. And we have to devise this mechanism in a way that they can be achievable and measurable at the end. All right, so that's the main perspective of that.

18
00:02:14.580 --> 00:02:28.559
MTS - A/V: So we can directly, I think, jump in. Yeah. Before that, this is a work ready call. So I'm going to do a quick roll call. In the room now we have Duane Wessels, we have Lars-Jo Hohn-Meet-Liman, we have Ken Renard, Frederick Lindberg.

19
00:02:28.620 --> 00:02:38.199
MTS - A/V: Jeff Osborn, Hunter Holan, Hero Hota, Wes Hardaker's in the room, and online from the RSAC, Afshan Abdul, Brad Bird.

20
00:02:39.310 --> 00:02:40.660
MTS - A/V: Peter Martin.

21
00:02:42.290 --> 00:02:48.549
MTS - A/V: and our chair is Naveed. So, Naveed, I will pass it back to you, and we'll get into the content.

22
00:02:50.270 --> 00:02:58.369
MTS - A/V: Yeah, thank you, Daniel, again. So we have a working document, if we can directly open that, and we.

23
00:02:58.370 --> 00:03:11.840
MTS - A/V: start over that document and, see where we go. So all the members, I are invited to put their comments on that, and that's.

24
00:03:11.840 --> 00:03:17.240
MTS - A/V: Since that's a work in progress, so the call stands as it is.

25
00:03:17.240 --> 00:03:30.009
MTS - A/V: So during this session or after that, if you have any comment to make on any of the criteria or indicators, please feel free to add that.

26
00:03:30.010 --> 00:03:54.899
MTS - A/V: Right. So this is, as a, recall or as a reference, this document is what we, developed along with, some of the volunteers from RSEC, when we were, doing the process, as a CIPCCG, right. So we presented to them an exemplary document.

27
00:03:54.900 --> 00:03:57.469
MTS - A/V: Of what kind of processes we have.

28
00:03:57.470 --> 00:04:04.599
MTS - A/V: And how can we measure them? And if we measure them, what kind of kind of indicators

29
00:04:04.600 --> 00:04:21.580
MTS - A/V: or measuring points, measuring data points we might have. This is in no way a fixed document. I expect that it would be a living document and could be matured as we move along this process.

30
00:04:21.620 --> 00:04:44.159
MTS - A/V: Right? So we can go over all those, or some of those measuring points in today's session, and I can open the floor for discussion for all of these, right? But before I move on to those, the document is shared in the

31
00:04:44.160 --> 00:05:03.190
MTS - A/V: in the draft agenda, but Daniel can share that in the chat as well. If you can go to the right, to the last page of this document, because I added some of the, you know, thoughts that I have

32
00:05:03.190 --> 00:05:12.320
MTS - A/V: for this process, and that could be a good starting point of discussion as well. Daniel, are you controlling this document? Okay.

33
00:05:15.020 --> 00:05:20.860
MTS - A/V: I think it's in the last page I added some of the thoughts that… yeah, this one, right?

34
00:05:20.910 --> 00:05:44.910
MTS - A/V: So, so, if, you see that, I, I just tried to, write down a few points of how I understand this process and what I, my experiences and learning, as, being part of the, the CIPCCD earlier, right? So we look for collecting, data, measuring data.

35
00:05:44.910 --> 00:06:00.399
MTS - A/V: then evaluating that data right? And eventually putting that in a cyclic process where we can achieve a continuous improvement. Right? So the the system has to be memory based.

36
00:06:00.450 --> 00:06:06.499
MTS - A/V: Means we consider what has happened in the past in order to.

37
00:06:06.780 --> 00:06:24.440
MTS - A/V: you know, to, to see where do we stand, next, right? So we, we go towards a systematic data collection as well as a systematic outcome assessment of those data points as part of this, this process.

38
00:06:24.440 --> 00:06:41.559
MTS - A/V: So I try to see how people do this process, or follow this process, and these are the six steps in the cycle that I identified, right? So, that is, first, you collect data.

39
00:06:41.560 --> 00:06:46.800
MTS - A/V: Then you keep monitoring your collection, right? You measure them, right?

40
00:06:46.800 --> 00:07:10.160
MTS - A/V: So you evaluate your data, that's the fourth step. You decide what to, what and how to post-process that data. And finally, you took action based on the decisions that you have, and you come back to the start of the cycle, right? So this is how the process would look like, right?

41
00:07:11.340 --> 00:07:30.269
MTS - A/V: So if we talk about the measuring points in the process, right? So measuring or data collection points could be our smart indicators. As we have discussed previously, they should be specific, they should be measurable.

42
00:07:30.270 --> 00:07:31.960
MTS - A/V: It should be achievable.

43
00:07:31.960 --> 00:07:56.840
MTS - A/V: They should be relevant to RSAC and they should be time bound means they could be achievable within a specific time. But in the process we can see that maybe not all of them can be that much specific or that much measurable, right? So we can also go towards the subjective kind of outcomes, like what is the feeling of a patient?

44
00:07:56.840 --> 00:08:12.280
MTS - A/V: of a survey that we hold, or what's the outcome of a particular session, or in the long run, what was the outcome of the actions that RSAC has taken over the months, or over the years, over the quarters?

45
00:08:12.280 --> 00:08:35.210
MTS - A/V: Right? So we can identify some measurement methods like how the data should be collected. It could be surveys, right? Part of annual survey that we already do, but along with that, some session-based survey that we perform, for example.

46
00:08:35.210 --> 00:08:44.190
MTS - A/V: Right? So how you measure the the sense of the success of a session.

47
00:08:45.200 --> 00:09:09.629
MTS - A/V: Right? Maybe the the consensus between the Rsos about something route. What is the feeling of that? Right? How do you evaluate the the consensus between root server operators? Eventually that is feeding into the Rsac right? The advisory review or updates about those how frequently we are updating and reviewing our process.

48
00:09:09.630 --> 00:09:13.330
MTS - A/V: That would be one of the points.

49
00:09:13.330 --> 00:09:33.819
MTS - A/V: Right? The outcomes of the working group, right? So what, is the overall feeling or the process that we follow, and how we are improving our processes for participation in the working group, for contribution to those working groups, and how successful are they, for example?

50
00:09:34.950 --> 00:09:58.240
MTS - A/V: And the review of the quarterly or annually defined goals by RSAC. So it could be short-term goals that we define quarterly-based or annual-based, right? It could be a 3-year base, so there could be different things that we can consider, right? So, we need to define some kind of benchmarks that the threshold that we would like to

51
00:09:58.240 --> 00:10:03.319
MTS - A/V: achieve going forward when we have such kind of indicators and we discuss them.

52
00:10:03.320 --> 00:10:27.449
MTS - A/V: And maybe the prioritization and weightage of each of these indicators that could potentially contribute to the criteria, right? And similarly, the weightage and priority of some criteria that could feed into the principles. So it's…

53
00:10:27.450 --> 00:10:30.110
MTS - A/V: For us as a group to decide.

54
00:10:30.110 --> 00:10:54.110
MTS - A/V: how much weightage or priority we would like to give to a particular indicator, right? So that's completely on us to decide how to proceed with that. In the start, we can consider only specific or subset of indicators or criteria to contribute, and we might say that something is 50% and the other is 30%.

55
00:10:54.110 --> 00:10:56.870
MTS - A/V: Contributing and things like that.

56
00:10:58.240 --> 00:11:11.440
MTS - A/V: Right? So so the objectives of this working party. One of the objectives is to develop the consensus based indicators which are adopted, which are balanced.

57
00:11:11.440 --> 00:11:21.909
MTS - A/V: Which can be integrated into regular operations easily without having too much, burden on RSAC or without having to change a lot of processes.

58
00:11:21.910 --> 00:11:46.610
MTS - A/V: And should be… they should be memory-based for constructive and continuous improvement, right? So, and then the final point is about the, phase-wise implementation of that. That was part of the… our discussion in the last call as well. Like, once we agreed as a working party, like, these are kind of indicators that we have, then the next step would be, and we are not sure

59
00:11:46.610 --> 00:12:05.319
MTS - A/V: Whether this is the remit of that, this working party or there's another working party needed for that, that comes up with the implementation plan of those recommendations that this outcome of this working party might have.

60
00:12:05.330 --> 00:12:30.029
MTS - A/V: So I just wanted to share these thoughts. They are not part of this document, but we can let it here if somebody wants to comment on that. But eventually, I just put them as to share with everybody, and I open the floor for anybody who has any feedback or comment of what I said or what I did.

61
00:12:30.030 --> 00:12:32.380
MTS - A/V: What my thoughts are in this,

62
00:12:32.430 --> 00:12:34.660
MTS - A/V: In in this page. Thank you.

63
00:12:42.310 --> 00:12:48.859
MTS - A/V: Naveed. First, st I'd like to say, thank you so much for the work you put into this. It's

64
00:12:49.200 --> 00:12:53.940
MTS - A/V: It's a tremendous effort, and it really really shows. And

65
00:12:55.470 --> 00:12:58.429
MTS - A/V: I came in here with a couple of concerns.

66
00:12:58.650 --> 00:13:04.960
MTS - A/V: that were just ameliorated by reading through the work you've already done.

67
00:13:05.820 --> 00:13:09.080
MTS - A/V: I'm re. I'm really impressed with how

68
00:13:09.840 --> 00:13:15.180
MTS - A/V: You skipped what would have been a really easy mistake to make, where the…

69
00:13:15.320 --> 00:13:19.909
MTS - A/V: The task of the RSAC is advice.

70
00:13:20.400 --> 00:13:32.579
MTS - A/V: to the community and the board, and so many people get that confused so easily with the operational things that Route Ops does. And this is entirely an advisory group.

71
00:13:32.600 --> 00:13:49.189
MTS - A/V: So I had lunch and was talking with my colleague Hans Petter here, and he was talking about, holy crap, there's just so many numbers of things. And I thought, I'll bet you anything it fell into how many root servers does it take to pile up a whatever and how many responses.

72
00:13:50.320 --> 00:14:03.670
MTS - A/V: And I can't figure out how you kept in the lane so well, but just I'm really impressed. I'm glad I wouldn't have to be the one to answer all these questions because a bunch of it is very detailed.

73
00:14:03.900 --> 00:14:08.790
MTS - A/V: But… Again, I'm really impressed having read through that, that you

74
00:14:09.060 --> 00:14:12.889
MTS - A/V: Got the assignment right, and did what, you know.

75
00:14:13.210 --> 00:14:16.349
MTS - A/V: Had to be done. So, well done, good job.

76
00:14:16.690 --> 00:14:30.190
MTS - A/V: Appreciate it. If nobody else has read the whole thing, I suspect other people might have feared what I was fearing, where this would have been really easy to fall into a bunch of questions about operational things. And it is, unless I missed a point.

77
00:14:30.380 --> 00:14:35.090
MTS - A/V: It is really, really just entirely, the

78
00:14:35.890 --> 00:14:40.010
MTS - A/V: part that you're gonna have to handle from now on, so this is your problem.

79
00:14:41.160 --> 00:14:45.660
MTS - A/V: Having said that, I wonder whether it couldn't be.

80
00:14:46.050 --> 00:14:52.659
MTS - A/V: condensed without harming how effective it looked like we were. The…

81
00:14:54.240 --> 00:14:57.980
MTS - A/V: ICANN is certainly prey to the thing of, nice job, it's long.

82
00:14:58.110 --> 00:15:08.740
MTS - A/V: And you know, the problem with long is, if it's something you're going to have to fill out later, somebody is going to dread doing this in about 3 years or 5 years or 10 years, because Jesus Christ, it's so long.

83
00:15:09.840 --> 00:15:19.680
MTS - A/V: What is your opinion on the ability to sort of make this more terse without harming how effective it looks to the rest of the organization? Is that a fair question?

84
00:15:20.760 --> 00:15:39.809
MTS - A/V: Yeah, thank you, thank you. Actually, One of the discussions we had in last working party session was that it is our choice as a group, RSAC, that we may

85
00:15:39.920 --> 00:15:49.000
MTS - A/V: Start it as a pilot project, and work on a subset of criteria and indicators, and even a subset of principles.

86
00:15:49.000 --> 00:16:12.170
MTS - A/V: For example, we say that, okay, you give us five principles to measure, but we would like to see how one or two principles would go with RSAC, right? And we will choose some kind of subset and some of criteria that measure them eventually through indicators, right? And once we are successful in doing that, then we are going to expand that and to see how we can incorporate that, right?

87
00:16:12.170 --> 00:16:21.740
MTS - A/V: So the second thing is, the whole cip process is actually composed of 6 years. Right? It it has 3

88
00:16:21.740 --> 00:16:33.810
MTS - A/V: 2 cycles of 3 years each right? And you may be collecting data because that 3 years has 3 phases, but often confused with 3 years.

89
00:16:33.810 --> 00:16:52.140
MTS - A/V: one year each, but it's not like this. It is, like, you might be collecting data for 2 years, right, or more than 2 years, and then you need to just analyze that and publish a report that could feed into the next cycle. But unless you complete 2 full cycles of 3 years, which is total 6 years.

90
00:16:52.140 --> 00:16:54.360
MTS - A/V: Right? Only then you…

91
00:16:54.360 --> 00:17:00.900
MTS - A/V: can decide on how things are going. Not before that. Right? So this is the the main purpose of cip.

92
00:17:01.790 --> 00:17:08.449
MTS - A/V: But just to be clear, are you saying, then, that a subset of the 5 with some subset of

93
00:17:08.730 --> 00:17:11.390
MTS - A/V: Questions to them. We could,

94
00:17:11.960 --> 00:17:14.269
MTS - A/V: Collect data for two years, put together

95
00:17:14.390 --> 00:17:19.209
MTS - A/V: Twice, without having to add back in or change what those were.

96
00:17:19.780 --> 00:17:38.789
MTS - A/V: Yeah, well, unless we measure all 5 of them, we can't say that we completed the CIP process, per se. But what I was suggesting is that maybe for, I mean, nobody's pushing us to start right away measuring everything, so we can measure, the first

97
00:17:38.880 --> 00:17:42.090
MTS - A/V: as one or 2 principles.

98
00:17:42.100 --> 00:18:04.179
MTS - A/V: and for one year or something right? And then we start the process itself for for that 6 year period, right where we we say that now we are comfortable with that. But eventually, once we you start the cycle of 3 years 1st cycle, which is composed of 3 years. You have to measure all 5 of them eventually, right beyond.

99
00:18:07.850 --> 00:18:10.650
MTS - A/V: It's about halfway to where I was hoping we could end up, but…

100
00:18:10.950 --> 00:18:16.950
MTS - A/V: Again, I think looking credible and, you know, the whole point of review is, This is sort of…

101
00:18:17.370 --> 00:18:19.819
MTS - A/V: Sustaining our, credibility.

102
00:18:20.820 --> 00:18:22.220
MTS - A/V: I can on the world at large.

103
00:18:22.870 --> 00:18:25.590
MTS - A/V: Any other questions or inputs?

104
00:18:28.160 --> 00:18:30.530
MTS - A/V: Hi, Lars Lieben here.

105
00:18:30.780 --> 00:18:31.580
MTS - A/V: I…

106
00:18:31.890 --> 00:18:41.460
MTS - A/V: I see a lot of criteria and measurements in here that strike me as very strange. I would like to see

107
00:18:42.280 --> 00:18:48.270
MTS - A/V: I either… Also, a column for how these are to be evaluated.

108
00:18:49.410 --> 00:18:52.400
MTS - A/V: These numbers. What's a good number, and what's a bad number?

109
00:18:54.570 --> 00:19:05.480
MTS - A/V: Or, or possibly and, to have these more clearly mapped to the mission and remit of RSAC.

110
00:19:07.840 --> 00:19:13.649
MTS - A/V: I can, I can take… One thing out of the Viteria said, the number of RSAC advisors published

111
00:19:14.800 --> 00:19:16.580
MTS - A/V: What's that got to do with it?

112
00:19:16.840 --> 00:19:21.269
MTS - A/V: The fewer things we publish, the better.

113
00:19:22.590 --> 00:19:26.770
MTS - A/V: Could be a view, because the fewer we publish, the fewer things we publish.

114
00:19:27.330 --> 00:19:34.030
MTS - A/V: means that there are less things to comment on, which means that things are running inside ICANN very well. We don't have to give advice.

115
00:19:36.600 --> 00:19:40.739
MTS - A/V: The fewer email messages we exchange, the better things work.

116
00:19:43.090 --> 00:19:46.360
MTS - A/V: The fewer things we do, the less it will cost.

117
00:19:48.910 --> 00:19:51.619
MTS - A/V: But we still have to fulfill our mission.

118
00:19:52.790 --> 00:19:58.959
MTS - A/V: So mapping it to to the mission is crucial to me. That's where we need to measure

119
00:19:59.350 --> 00:20:12.270
MTS - A/V: How do we fulfill our role and what we do about it? And how can we do that at the most cost effective way, both for ICANN as an organization, but also for ourselves with our time?

120
00:20:13.810 --> 00:20:22.619
MTS - A/V: So how can we avoid spending resources for ICANN? How can we avoid spending resources for ourselves and producing volume

121
00:20:22.800 --> 00:20:24.419
MTS - A/V: uses of resources.

122
00:20:24.840 --> 00:20:29.740
MTS - A/V: So, to find the balance, and how to measure that balance, is…

123
00:20:30.230 --> 00:20:33.459
MTS - A/V: The sweet spot that we want to reach, in my view.

124
00:20:33.860 --> 00:20:34.600
MTS - A/V: Thanks.

125
00:20:36.090 --> 00:20:50.939
MTS - A/V: Yeah, thanks, Liman. That's a valid point. This, I mean, to see if an S or AC is fulfilling its purpose.

126
00:20:51.460 --> 00:21:02.829
MTS - A/V: The first thing we have to answer is whether they are doing what they are supposed to do. For example, I mean, okay, you are right in terms of number of advisories like

127
00:21:02.830 --> 00:21:19.540
MTS - A/V: that we publish or not, but I mean, we have to see it in the ICANN overall structure, because this is what the community would like to see, right? So, if there is an SOAC, right, and it is not producing anything that it's supposed to be, for example.

128
00:21:19.540 --> 00:21:26.229
MTS - A/V: then would it fit in the overall purpose, or or I mean, this is not

129
00:21:26.360 --> 00:21:37.720
MTS - A/V: a question in itself, but a thought that I'm putting forward. So, in that context, the advisories published would still be relevant. Now, how much?

130
00:21:37.720 --> 00:22:02.600
MTS - A/V: is a separate question, right? Because whenever we are asked to do an advisory from board, we have to produce that, right? So otherwise, I mean, but we can also comment on things, for instance, that could also be part of this fulfilling purpose. For example, there is some public comment, and we feel that it is relevant to us, RSAC could also do that. So that could be

131
00:22:02.600 --> 00:22:19.919
MTS - A/V: part of the indicators going forward as well, right? But there is one word that I used is called, is what is benchmarking. That is what I look towards. This is what the column that you mentioned, right? Because going forward, it is very important

132
00:22:19.920 --> 00:22:24.330
MTS - A/V: Whatever we see, we need to decide, right, how

133
00:22:24.330 --> 00:22:49.160
MTS - A/V: is it is going to be measured. And if we think that and evaluate it eventually. So if we think that we can't evaluate it, evaluate that in the as part of the CIP, then we let it go right. This is how the approach would be right when we created this document. These were the suggestions by some of the the members.

134
00:22:49.160 --> 00:22:52.670
MTS - A/V: Right? For instance, and we put that to the…

135
00:22:52.670 --> 00:23:04.869
MTS - A/V: to whole RSEC group to comment and to, you know, update that. But this is what we had at that time. That's why we are starting from this document as of now.

136
00:23:06.310 --> 00:23:10.389
MTS - A/V: But that would be a good discussion to to have going forward.

137
00:23:22.350 --> 00:23:23.690
MTS - A/V: It's Hans, not Mike.

138
00:23:29.320 --> 00:23:35.360
MTS - A/V: I just had it. Sorry. Hans Petterholing, RIPENCE CAKE ROOT.

139
00:23:35.620 --> 00:23:40.200
MTS - A/V: So if I understand correctly, these five principles are mandatory.

140
00:23:40.400 --> 00:23:46.150
MTS - A/V: So then they're non-negotiable. So we need to measure success for each of these principles.

141
00:23:46.310 --> 00:23:54.910
MTS - A/V: And the document, I did a quick count, it's at least 60 indicators, or some of them contains 1, 2, 3, so we're talking maybe…

142
00:23:55.770 --> 00:24:10.369
MTS - A/V: 80 plus numbers, right? I think that is, order of magnitude too high. How can we measure success with identifying, let's say, 3 indicators for each of these principles?

143
00:24:10.570 --> 00:24:15.390
MTS - A/V: That are meaningful, and… You know that I think that's you know.

144
00:24:15.520 --> 00:24:24.610
MTS - A/V: around best practice for some of the methods I use for measuring success or continuous improvement of processes is to keep it as

145
00:24:24.740 --> 00:24:36.590
MTS - A/V: You know, 3 to 5 high-level objectives, and maximum 3 key results for each objective. That brings us to, you know, 15 seems a lot, but still, that's manageable, I think.

146
00:24:36.980 --> 00:24:52.399
MTS - A/V: But reading this, it's really difficult to go through and say that we don't need this one because and then what is really the key one? So I think maybe that's a useful exercise for everybody to look for each of these criteria. What's the top one that needs to be there?

147
00:24:52.680 --> 00:24:58.550
MTS - A/V: And then you probably can't decide on one, but it will be two or three, right? And do we converge on that?

148
00:25:01.710 --> 00:25:05.520
MTS - A/V: Yeah, thanks, Hans and Naveed here.

149
00:25:05.570 --> 00:25:17.199
MTS - A/V: Alright. So what the the cip suggest in its or recommends in its document is that all these principles should have at least 3 to 5 criteria.

150
00:25:17.230 --> 00:25:27.010
MTS - A/V: right? But then, how would you measure that criteria? There could be number of indicators. There might be one for each criteria. There might be more than one.

151
00:25:27.010 --> 00:25:38.129
MTS - A/V: So actually, as you said, right, so we look for at least 15 criteria that we have to fix first, at least 15 more than that.

152
00:25:38.130 --> 00:25:44.889
MTS - A/V: And then we will see how they can be measured, but tho- for those f- 15 we might have.

153
00:25:44.890 --> 00:26:09.519
MTS - A/V: I mean 30 to 40 indicators. And in this document, if once I think there are some indicators that are contributing to more than one principles as well. So there are some overlaps. For example, you measure one thing and you say that, okay, this is you get more members. This is contributing to fulfilling your purpose.

154
00:26:09.520 --> 00:26:16.830
MTS - A/V: for example, as well as the community engagement and all right. So there are some overlaps in this document as well.

155
00:26:17.090 --> 00:26:25.159
MTS - A/V: So, I think you misunderstood me. I was aiming for a maximum of 15 indicators, not 15 per principles.

156
00:26:26.320 --> 00:26:40.230
MTS - A/V: Yeah, I I get you. But what I'm saying is the document CIPCCG document right, which is already in place mandates that we should have at least 3 criteria per principal.

157
00:26:40.230 --> 00:26:55.139
MTS - A/V: right? If you decide to measure one criteria by one indicator, then you can have 15 right? But if there would be some criteria that would require more than one. So I I don't see that we will end up having

158
00:26:55.140 --> 00:27:03.710
MTS - A/V: 15. But, as I said, once we run it as a pilot, we can go for a subset, and to see how things are going right.

159
00:27:08.340 --> 00:27:09.550
MTS - A/V: Liman, please.

160
00:27:10.690 --> 00:27:12.790
MTS - A/V: Thanks, Liman. Here again.

161
00:27:13.020 --> 00:27:15.300
MTS - A/V: Two more comments.

162
00:27:15.810 --> 00:27:22.749
MTS - A/V: I think we shouldn't necessarily… We should back up and and see what we

163
00:27:23.260 --> 00:27:28.589
MTS - A/V: want to evaluate for quality, because I… my sense is that we want to

164
00:27:28.850 --> 00:27:32.730
MTS - A/V: Make sure that we have the right quality of the of the work in the group.

165
00:27:33.360 --> 00:27:37.510
MTS - A/V: And… Quality isn't necessarily mentioned by numbers.

166
00:27:37.950 --> 00:27:40.540
MTS - A/V: Some cases it can be, but not always.

167
00:27:40.690 --> 00:27:44.080
MTS - A/V: So… We, we should also

168
00:27:44.860 --> 00:27:53.159
MTS - A/V: look at these and say, can this be measured in a different way? That gives us a better output and a better understanding of

169
00:27:53.500 --> 00:27:54.810
MTS - A/V: of the quality.

170
00:27:55.460 --> 00:28:01.880
MTS - A/V: And then a totally different question is, Well, this CIP instruction document

171
00:28:02.480 --> 00:28:14.950
MTS - A/V: Does it also come with a financial plan for how to pay for all the work that needs to go into all the SOACs for doing all of this, setting this up, doing the measurements, evaluating and everything for

172
00:28:15.310 --> 00:28:18.230
MTS - A/V: 3 to 5 times 5 criteria.

173
00:28:18.390 --> 00:28:21.059
MTS - A/V: That's a lot of money.

174
00:28:24.140 --> 00:28:30.299
MTS - A/V: Yeah, I'm not the right person, maybe to answer this. But

175
00:28:30.300 --> 00:28:54.709
MTS - A/V: I mean, right? That was a rhetorical question. Yeah. But I think we discussed in the last meeting that it might require a tool to be engaged as well with CIP documents suggest that, right? And we should see some kind of planning of engaging those softwares and those measuring tools that automatically

176
00:28:54.740 --> 00:29:19.360
MTS - A/V: does this workflow, right? So that I have no idea at what point we will do. About the data collection and measuring them, I mean, absolutely right. For example, here in the points that I showed before, I said that there could be some indicators that would be smart in

177
00:29:19.360 --> 00:29:43.800
MTS - A/V: That SMART kind of, but there could be subjective based, right? So that gives an overall quality assessment, but not quantitatively, right? It gives a qualitative assessment of the things, right? And you can have the same data point that contributes partially to the quantitative one and partially to the qualitative one. For example, the advisories that you published.

178
00:29:43.910 --> 00:29:53.610
MTS - A/V: You can have number of advisories, but you can also get the quality of those advisories, right? Somehow, that we have to see how

179
00:29:53.610 --> 00:30:13.380
MTS - A/V: it could be done. You do an engagement with the community. Now, that could not be assessed as, you know, as a number of things, but you can get a survey of the audience, for example, and get their satisfaction level in that. So that is also a qualitative kind of

180
00:30:13.380 --> 00:30:17.300
MTS - A/V: assessment in the process right rather than a quantitative one.

181
00:30:17.300 --> 00:30:33.660
MTS - A/V: So, of course, the, the, this, especially the feedback of the survey that we, different surveys that we, we get as part of the process, should, have that, thing ticked, right? So, that's how I see it.

182
00:30:33.740 --> 00:30:34.430
MTS - A/V: Cube.

183
00:30:39.810 --> 00:30:41.399
MTS - A/V: Any other comment?

184
00:30:41.580 --> 00:30:42.310
MTS - A/V: So…

185
00:30:50.120 --> 00:30:54.880
MTS - A/V: We have Hans from Hans Petter, and then down the end, we have Christian from ICANN Org, who might be able to…

186
00:30:55.610 --> 00:30:56.810
MTS - A/V: So, Christian?

187
00:30:58.770 --> 00:31:01.200
MTS - A/V: Thank you. Christian Wheeler from ICANN Org.

188
00:31:01.470 --> 00:31:15.899
MTS - A/V: I'm part of the CIP team, that's kind of helping, monitor and, and, provide guidance to all the SOACs implementing CIP, so really glad to see that this is being worked on in RSAC, and that a lot of

189
00:31:16.010 --> 00:31:18.249
MTS - A/V: Great thought has been put into this framework.

190
00:31:18.440 --> 00:31:22.769
MTS - A/V: I just wanted to to echo a lot of what what has been said about

191
00:31:22.770 --> 00:31:42.130
MTS - A/V: I understand, you know, the need for flexibility. That is kind of the goal of the CIP framework, is so that each SOAC has some flexibility while still having an overall framework by which all of the SOACs can be looked at when looking holistically about all of their various frameworks, but granted, there will be some variation there.

192
00:31:42.130 --> 00:31:48.740
MTS - A/V: And each SOAC has been tackling it a little bit differently when it comes to their criteria and indicators.

193
00:31:48.740 --> 00:31:56.700
MTS - A/V: What I would suggest, just from my own observations across the other SOACs, is, the…

194
00:31:56.890 --> 00:32:07.989
MTS - A/V: Phase 1 is where, which is where you guys are right now, is the most important, I would say, in terms of scoping and making sure that the indicators and criteria are manageable.

195
00:32:08.130 --> 00:32:15.780
MTS - A/V: With the understanding that while there may not be, you know, the need for You know, 3 indicators.

196
00:32:15.860 --> 00:32:21.520
MTS - A/V: For every single criteria, what's more important is the quality of that data.

197
00:32:21.520 --> 00:32:38.000
MTS - A/V: and how well it is applicable to what RSAC needs in terms of looking at its efficiencies and improvements to be made. So, I would suggest not having indicators, just for having indicators' sake, you know, having the… for the minimum 3.

198
00:32:38.000 --> 00:32:47.749
MTS - A/V: If that can be achieved in less, that is… that can be done. I would recommend not abusing that, as far as just having a… becoming a checkbox exercise.

199
00:32:47.750 --> 00:33:00.309
MTS - A/V: But for instance, looking at some of the… some of the indicators, it's just good to bear in mind what is… first of all, what is even possible to gather? What's reasonable to gather? I know some other SOACs, you know, have…

200
00:33:00.450 --> 00:33:15.570
MTS - A/V: tons of indicators of all of a wish list of everything that they want to see, not all of which is actually possible to gather or would take a lot of manual work or updates to the ICANN Org staff's daily tasks when it comes to

201
00:33:15.570 --> 00:33:28.030
MTS - A/V: tracking some of this data. So really, I would just recommend focusing on what is actually applicable most to the criteria, what data would be most useful to the RSAC to have to make sure that it is

202
00:33:28.040 --> 00:33:34.260
MTS - A/V: on track with its tasks and fulfilling its purpose. And you know, for the various 5 principles.

203
00:33:34.310 --> 00:33:43.470
MTS - A/V: So just wanted to highlight that, and just want to say good job, and also recognizing as well that the criteria and indicators can overlap.

204
00:33:43.710 --> 00:33:50.560
MTS - A/V: So that is… that'll also be useful to help kind of reduce some of that time, particularly that staff would need to do to gather some of that data.

205
00:33:50.870 --> 00:33:51.540
MTS - A/V: Thanks.

206
00:33:54.680 --> 00:33:56.340
MTS - A/V: Yeah, thank you.

207
00:33:56.750 --> 00:33:58.930
MTS - A/V: Anything else from anybody.

208
00:33:59.650 --> 00:34:01.100
MTS - A/V: Is there anybody in queue?

209
00:34:03.910 --> 00:34:11.639
MTS - A/V: Okay, so How much time we have Daniel for today? Is there anything on on the agenda?

210
00:34:11.880 --> 00:34:12.920
MTS - A/V: After this.

211
00:34:14.100 --> 00:34:20.530
MTS - A/V: Yes, actually, Dan Gluck, I'd like to actually talk about some of…

212
00:34:20.929 --> 00:34:26.420
MTS - A/V: Are other organizational-wide goals that we might need to incorporate into

213
00:34:26.670 --> 00:34:36.280
MTS - A/V: our, continuous improvement program. I am pulling up the agenda here. And that'd be our, Workstream 2 update, if that's okay for me to proceed with now.

214
00:34:36.620 --> 00:34:39.329
MTS - A/V: Sure, sure, good. So.

215
00:34:39.520 --> 00:34:51.230
MTS - A/V: Going back a long time, there was a community group, came up with Workstream 1 and Workstream 2 recommendations. Workstream 2 recommendations are still open for most of the SOs and ACs to implement.

216
00:34:51.600 --> 00:35:00.609
MTS - A/V: It has been recommended to us, to implement recommendations 1.2, 1.3, 1.4, and 1.5.

217
00:35:00.730 --> 00:35:04.520
MTS - A/V: From… Pull up the right document.

218
00:35:30.480 --> 00:35:31.460
MTS - A/V: Here we are.

219
00:35:33.770 --> 00:35:45.870
MTS - A/V: So yeah, there are… these are some of the, recommendations to improve diversity. These are standard throughout the different SOs and ACs in terms of goals that we need to accomplish.

220
00:35:46.050 --> 00:35:53.030
MTS - A/V: And it was suggested that we could potentially fold these into some of the continuous improvement program recommendations.

221
00:35:53.200 --> 00:35:56.000
MTS - A/V: And indicators, that we have there.

222
00:35:56.520 --> 00:36:07.780
MTS - A/V: A lot of this, I think, the way Christian summed it up to me was there's a survey that the RSAC could distribute to, the RSAC and the RSAC Caucus.

223
00:36:08.150 --> 00:36:13.199
MTS - A/V: And, then based on the results of that survey, we'd be able to

224
00:36:13.390 --> 00:36:23.199
MTS - A/V: meet our understanding of the structures and the people that are involved in them, and then, try to, I guess.

225
00:36:23.360 --> 00:36:31.620
MTS - A/V: prove, I guess, our… Practices and publications to encourage a.

226
00:36:32.370 --> 00:36:34.789
MTS - A/V: You know, wider swath of people to…

227
00:36:35.200 --> 00:36:42.719
MTS - A/V: become involved with the caucus side of things, because that's… the RSAC's not open, but the RSAC caucus, is

228
00:36:43.140 --> 00:36:44.630
MTS - A/V: To…

229
00:36:44.900 --> 00:36:55.020
MTS - A/V: applicants that aren't attached to an RSO. So, we have these four recommendations here. I've tried to see if they could fit within…

230
00:36:55.370 --> 00:36:57.389
MTS - A/V: Our working document…

231
00:37:00.430 --> 00:37:01.989
MTS - A/V: Pull that up here.

232
00:37:07.040 --> 00:37:12.950
MTS - A/V: Adding to the, number of… Indicators that we have, but…

233
00:37:13.240 --> 00:37:16.370
MTS - A/V: Yeah, the just a couple of statements here and

234
00:37:17.010 --> 00:37:22.540
MTS - A/V: fits into the idea that RSAC has a mechanism to efficiently allocate and use of.

235
00:37:22.860 --> 00:37:26.450
MTS - A/V: Available human resources as per the opportunities to contribute.

236
00:37:27.160 --> 00:37:31.949
MTS - A/V: the three goals here. I don't really think there's anything

237
00:37:32.740 --> 00:37:40.219
MTS - A/V: Now, to discuss, because there really hasn't been a lot shared about this in recent times, and no one has an opportunity to

238
00:37:40.390 --> 00:37:41.769
MTS - A/V: Read all of this yet?

239
00:37:41.910 --> 00:37:44.910
MTS - A/V: Just something for us to consider going forward.

240
00:37:45.220 --> 00:37:51.610
MTS - A/V: our, our Inclusion of these efforts of the Workstream 2 recommendations.

241
00:37:51.760 --> 00:37:54.750
MTS - A/V: Potentially into our continuous improvement program.

242
00:37:54.960 --> 00:37:56.389
MTS - A/V: work as well.

243
00:37:56.590 --> 00:37:58.190
MTS - A/V: And Christian has a hand.

244
00:37:59.910 --> 00:38:00.870
MTS - A/V: Thanks, Dan.

245
00:38:01.220 --> 00:38:14.829
MTS - A/V: Yes, so there is, a good potential for overlap here for the Workstream 2 recommendations, and these came out in 2018, which included a lot of recommendations both for ICANN Org and for the SOACs.

246
00:38:14.910 --> 00:38:34.089
MTS - A/V: And to date, the SOACs have not yet fully implemented Workstream 2 Recommendation 1, which is about diversity, and so it is this 0.2, 0.3, 0.4, 0.5, that still need to be implemented, largely across the community, and

247
00:38:34.520 --> 00:38:40.740
MTS - A/V: It's actually pretty. I would say it's pretty simple to implement, I believe, because

248
00:38:40.830 --> 00:38:55.140
MTS - A/V: 1.2 is just to do a, look through the various working documents, charters, operating procedures, to see where are these, seven diversity elements, mentioned, if at all.

249
00:38:55.140 --> 00:39:08.070
MTS - A/V: And staff has already kind of done a review of that, and from what I've seen there, it isn't mentioned in any of them, but that… once that's kind of completed, you know, as a review, just formally, then say, okay, 1.2 is done.

250
00:39:08.100 --> 00:39:11.029
MTS - A/V: 1.3 is the actual diversity survey.

251
00:39:11.060 --> 00:39:29.960
MTS - A/V: Which has… a template has already been drafted by staff. It would just need to be sent, and obviously amended some of the questions if needed. And then point four is publishing it and coming up with some kind of, objectives to increase diversity if necessary, if relevant.

252
00:39:30.090 --> 00:39:44.140
MTS - A/V: And point 1.5 is about, kind of committing to doing it once every, no later than one every three years, which is… aligns with the CIP cycle as well, which also happens every three years. So if this was just built into the.

253
00:39:44.860 --> 00:39:47.580
MTS - A/V: the CIP cycles.

254
00:39:47.660 --> 00:39:48.790
MTS - A/V: then…

255
00:39:48.800 --> 00:40:08.359
MTS - A/V: this is already a good source of data that could be used, whether it's for principle, you know, 4 or 5, there could be some overlap as far as being accountable internally to stakeholders and substructures, or fulfilling the mission of ICANN, that sort of thing. So, I mean, there's definitely ways to incorporate that, since it's already an exercise that needs to be done.

256
00:40:08.360 --> 00:40:17.729
MTS - A/V: And it's about data gathering to use some of that data. And then once it's in the CIP and you say, yes, we've done the survey, that's a check. And you have that indicator.

257
00:40:17.810 --> 00:40:20.519
MTS - A/V: So, there is some good opportunity there to

258
00:40:20.820 --> 00:40:35.580
MTS - A/V: kind of take credit for both and knock out 2 birds with one stone, and, as I said, the survey is already done. It just needs to be sent out and published. And so there is some. It's I would consider it some pretty low hanging fruit. Granted, it does have some

259
00:40:35.900 --> 00:40:36.840
MTS - A/V: of

260
00:40:37.090 --> 00:40:43.120
MTS - A/V: people have different feelings about diversity and whether they want to publish that kind of data. So that's obviously a conversation that needs to be had.

261
00:40:43.190 --> 00:40:48.839
MTS - A/V: But there is some good overlap here that could, be used well for the CIP.

262
00:40:48.850 --> 00:41:00.409
MTS - A/V: Bearing in mind that you don't want to have too many indicators and be overwhelmed with too much stuff. So, as a reminder, too, the CIP is, right now, is the first cycle. It's very much a pilot.

263
00:41:00.410 --> 00:41:10.079
MTS - A/V: Right now, you don't need to have everything under the sun, and it's the understanding that with each cycle, the criterion indicators could be updated. So…

264
00:41:10.160 --> 00:41:12.099
MTS - A/V: Don't feel like you need to…

265
00:41:12.420 --> 00:41:22.809
MTS - A/V: can boil the ocean here. So, just wanted to flag that, because this could be a good opportunity to knock this out, which has been on the community's plate for quite some time.

266
00:41:23.250 --> 00:41:23.950
MTS - A/V: Thanks.

267
00:41:26.730 --> 00:41:28.310
MTS - A/V: Christian, just to

268
00:41:28.430 --> 00:41:38.130
MTS - A/V: confirm when you say each cycle. There have been a bunch of numbers of years called cycle. Do you mean every one year, 3 years, 6 year.

269
00:41:38.370 --> 00:41:42.739
MTS - A/V: The CIP is imagined to be a 3-year cycle.

270
00:41:42.870 --> 00:41:59.460
MTS - A/V: How they, how they set it up in the framework, just as a matter of trying to be, flexible and overall is just kind of a broad swath that it'd be one year for… you know, for creating the criterion indicators, one year for implementing improvements, and then one year for reporting.

271
00:41:59.590 --> 00:42:02.520
MTS - A/V: Granted on the ground. It's probably not gonna

272
00:42:02.740 --> 00:42:11.779
MTS - A/V: align that, you know, that quickly, especially with the fact that these are continuous improvements. Improvements are happening all the time. That's kind of why we're really hoping that

273
00:42:11.970 --> 00:42:22.089
MTS - A/V: For at least for this first cycle, which ends this first three-year CIP cycle, which ends July 2028, is when a progress report would need to be published.

274
00:42:23.980 --> 00:42:29.470
MTS - A/V: The Workstream 2 recommendation, I think it's 1.5 or 1.6.

275
00:42:29.930 --> 00:42:34.470
MTS - A/V: Is that a diversity survey would be conducted by each soac.

276
00:42:34.830 --> 00:42:42.119
MTS - A/V: no later than, you know, once every 3 years. So, that could align with the CIP if they wanted to incorporate that as part of the CIP.

277
00:42:43.570 --> 00:42:44.969
MTS - A/V: Thank you. That cleared it up.

278
00:42:46.620 --> 00:43:00.680
MTS - A/V: And also to clarify, too, that the progress report is something that gets published for public comment, and as you noted, you don't need to… it is kind of like a snapshot point in time. This is, you know, basically saying, this is our criterion indicators.

279
00:43:00.680 --> 00:43:15.300
MTS - A/V: this is what we have to date, these are the improvements that we've identified and prioritized, and these are where we're at with the ones that we've prioritized. It doesn't… you don't need to complete all the improvements. You can say we've, you know, we're focusing on principles 1 and 2 for this first cycle.

280
00:43:15.300 --> 00:43:32.200
MTS - A/V: And for the next cycle, we'll focus on the next three, or what have you. So it is flexible, and it is not meant to be something that you need to complete everything by July 2028. It is just kind of a point in time, just to make sure that everyone is kept on making progress toward continuous improvement, which

281
00:43:32.200 --> 00:43:37.770
MTS - A/V: all the SOACs are already always doing, so this is just kind of a matter of taking credit for what work you're doing.

282
00:43:42.930 --> 00:43:43.860
MTS - A/V: Hans. Better.

283
00:43:46.390 --> 00:43:50.040
MTS - A/V: So Hans Petter ripens the sea still.

284
00:43:52.260 --> 00:44:01.580
MTS - A/V: The survey sounds like a low-hanging fruit, right? You said it's already done. Could you share what's in that survey, at least on a high level?

285
00:44:03.970 --> 00:44:19.959
MTS - A/V: Sure, I don't have the document pulled up in front of me, but it is questions that align with those seven elements of diversity. I believe there's two surveys that staff had drafted, one that is

286
00:44:20.070 --> 00:44:24.770
MTS - A/V: kind of following those those 7 elements which ones are applicable to you. That sort of thing.

287
00:44:24.830 --> 00:44:44.270
MTS - A/V: And then there was another survey, if the community would prefer to have a different kind of range of data that would be more… that's more aligned with, barriers to participation, to see, like, are there… are you experiencing any barriers? How are they linked to, you know, your, perspective as a community member and your.

288
00:44:44.570 --> 00:45:01.619
MTS - A/V: whatever diversity elements are applicable to you. So there are two surveys drafted. You could combine them into one, or you could do them separately. It doesn't… it's not required that you do two. It's just that a survey that's basically in line with

289
00:45:02.380 --> 00:45:13.089
MTS - A/V: the diversity elements, and which ones are applicable to your community group. And the first step of that, 1.2, I believe, is about what is mentioned, what

290
00:45:13.230 --> 00:45:32.700
MTS - A/V: if any, elements of diversity or requirements of diversity are noted in your operating documents, which I don't believe there is any at this point, so it's not a requirement. that you need to have it as part of it, it's just kind of a fact-gathering mission, which I believe staff has already kind of done, and I can share that with whoever I need to.

291
00:45:32.800 --> 00:45:47.129
MTS - A/V: Follow-up question to that. We already do a survey once a year for the RSAC caucus, so if we'd been, you know, one month earlier, we couldn't have incorporated some of that in this survey, which would be, you know, a really low-hanging fruit.

292
00:45:47.150 --> 00:46:02.579
MTS - A/V: Especially the… and maybe we already have questions there on barrier for participation, right? Because I think some of this we have geographic diversity, we already measure, for instance. So for the RSAC caucus, this could make some sense.

293
00:46:02.590 --> 00:46:18.049
MTS - A/V: For RSAC, it's much more difficult because we're appointed by our own serving organizations. And, you know, if we're going to introduce gender diversity, then, well, half of us will have to go home and change to one of our female colleagues.

294
00:46:18.050 --> 00:46:23.790
MTS - A/V: which will then be a difficult decision to choose which half of the organization. And I mean.

295
00:46:24.560 --> 00:46:28.729
MTS - A/V: I see the greater good there, but I find it very difficult in an…

296
00:46:29.100 --> 00:46:36.449
MTS - A/V: in a body like, like RSAC, where it's representing individual organizations, right? So…

297
00:46:36.690 --> 00:46:47.150
MTS - A/V: Yeah. So mapping it out and having it survey and reporting on it. Sure. But then once we have those numbers, what do we want to do with them? Big question mark.

298
00:46:47.840 --> 00:47:07.660
MTS - A/V: RSAC caucus, or we could make a drive to… I think we identify some regions that we lack participation for, so we could do outreach there. And there may be other diversity factors that could also be, you know, interesting to think about. That may mean changing the criteria we have in some manners, because

299
00:47:07.700 --> 00:47:10.659
MTS - A/V: We're focusing on a very narrow skill set.

300
00:47:10.940 --> 00:47:18.009
MTS - A/V: And if it's diversity in skills, then, you know, we may need to have different criteria for skill sets, right? So, yeah.

301
00:47:22.120 --> 00:47:28.869
MTS - A/V: Just to speak for that, I spent, I forget how many years, as the chairman of the membership committee, and in practice.

302
00:47:29.060 --> 00:47:39.350
MTS - A/V: although not in documentation. When we received an application from somebody in an area that was underrepresented.

303
00:47:39.570 --> 00:47:49.199
MTS - A/V: the… The universal thinking among the membership was that the criteria should not be as strict.

304
00:47:49.440 --> 00:47:57.939
MTS - A/V: For a variety of reasons. And similarly, the requirement was stricter in overrepresented areas.

305
00:47:57.990 --> 00:48:09.780
MTS - A/V: So documenting that I think would be challenging. But it was absolutely something where, as you're saying, there are, you know, this is 12 companies that are 12 individuals. It's kind of hard to

306
00:48:09.890 --> 00:48:12.989
MTS - A/V: To enforce how you end up with.

307
00:48:13.170 --> 00:48:27.539
MTS - A/V: with some kind of imagined good. The caucus is a lot easier and it's probably the place where you could document it if you wanted to. But just to say that the practice of the last few years was.

308
00:48:27.650 --> 00:48:39.349
MTS - A/V: was actively, you know, trying to seek greater geographic and God knows the gender diversification is the ongoing technical problem for 40 plus years I've been doing this.

309
00:48:40.100 --> 00:48:43.459
MTS - A/V: You know, you can try, but there's only so much you can get to.

310
00:48:51.800 --> 00:48:55.089
MTS - A/V: Sorry, I'm… just want to

311
00:48:55.340 --> 00:49:01.489
MTS - A/V: agree with the notion as far as that there is a survey already being done for the RSAC caucus, I would encourage

312
00:49:01.730 --> 00:49:06.930
MTS - A/V: incorporating into current practices as much as possible, including with the CIP.

313
00:49:07.210 --> 00:49:13.040
MTS - A/V: Just because I'm noting that the community has a lot on their plates, and so…

314
00:49:13.170 --> 00:49:26.930
MTS - A/V: Obviously, we don't want to… to add too much burden, and so that's why, especially with this first cycle, to incorporate as much work as you're already doing, improvements that you're already working on or know about, and…

315
00:49:27.760 --> 00:49:39.960
MTS - A/V: taking… getting credit for what you're already doing, without having to create too much, you know, without requiring any kind of changes to processes out the gate. It's really just about trying to.

316
00:49:40.120 --> 00:49:42.609
MTS - A/V: See how much, how many birds you can get with one stone.

317
00:49:42.770 --> 00:49:44.400
MTS - A/V: so I'd encourage that as much as possible.

318
00:49:47.870 --> 00:49:51.620
MTS - A/V: So, commenting on that list, we already do geographic region.

319
00:49:51.900 --> 00:50:07.079
MTS - A/V: I'm not sure how to interpret regional representation, but we already have geography. I saw a pie chart on that. That's easy. Language, I don't think we track that. We have English as a working language, so we haven't ever kind of thought about that as…

320
00:50:07.390 --> 00:50:08.700
MTS - A/V: as necessary.

321
00:50:08.840 --> 00:50:23.809
MTS - A/V: Gender, we don't track. It could be interesting to map out, but you can see how many women we have in this room. Not many today. Usually we have a couple, but it's very imbalanced. Age, interesting.

322
00:50:23.860 --> 00:50:36.200
MTS - A/V: Physical disability, that's a health data. That's not something that I'm necessarily willing to even share or know letters stored. So I think I can need to be very careful with

323
00:50:36.600 --> 00:50:39.100
MTS - A/V: You know.

324
00:50:39.270 --> 00:50:42.169
MTS - A/V: For what purpose are they asking that question, right?

325
00:50:42.450 --> 00:50:47.889
MTS - A/V: For assistant at the meeting, and so on, for sure, but… Yeah,

326
00:50:48.220 --> 00:50:52.049
MTS - A/V: I don't know if that's something that the RSAC wants to…

327
00:50:52.600 --> 00:50:56.100
MTS - A/V: to store for participating in virtual meetings.

328
00:50:56.490 --> 00:51:03.839
MTS - A/V: Diverse skills. Yeah, that's back to what I said. We have requirements of having knowledge about DNS and

329
00:51:03.960 --> 00:51:07.869
MTS - A/V: understanding root server operations. We do get

330
00:51:08.190 --> 00:51:12.490
MTS - A/V: Applications that have very diverse skills.

331
00:51:12.910 --> 00:51:22.109
MTS - A/V: But they're missing the core understanding of that, right? And that also reflects on stakeholder groups and constituency. We don't ask that questions, because that's…

332
00:51:22.670 --> 00:51:31.739
MTS - A/V: kind of irrelevant on whether somebody with those skills identify as a numbers or a names person, or academia, or whatever.

333
00:51:32.420 --> 00:51:35.900
MTS - A/V: So, how relevant this is for us?

334
00:51:36.450 --> 00:51:37.570
MTS - A/V: I don't know.

335
00:51:38.140 --> 00:51:40.839
MTS - A/V: Yeah, if I can just

336
00:51:41.620 --> 00:52:01.339
MTS - A/V: comment on that. This is Naveed. I think one of the idea of work stream to recommendation, this recommendation is that it mandates each SOAC to identify which aspects are relevant to them, and not to incorporate all of them.

337
00:52:01.340 --> 00:52:06.649
MTS - A/V: So I don't think that it's a requirement that we have to fulfill all seven of them.

338
00:52:06.650 --> 00:52:31.140
MTS - A/V: It is, I think the exercise would be to identify which are relevant to them and how we are, are doing, there, right? So, and, I think collecting data for all of them, would still be, or can still be done, but then, as a conclusion or as a valuation, RSAC can always put into place that these are the.

339
00:52:31.220 --> 00:52:48.860
MTS - A/V: the aspects of diversity that we are, we are incorporating, and these are the, the reasons for that. I think for, for the disability, I think in the survey, you can always, I mean,

340
00:52:49.180 --> 00:52:52.830
MTS - A/V: ask the the the participant.

341
00:52:52.830 --> 00:53:17.269
MTS - A/V: to reveal if they want to, and that gives a kind of, you know, a knowledge to our SAC that we have someone that has kind of disability, right, or an aspect of that. It is no way, right, mandating them or requiring them to do that, but you can incorporate that as part of the survey.

342
00:53:17.270 --> 00:53:30.339
MTS - A/V: But my point there is that under certain countries' privacy laws, you need to have a ground for collecting that data. And I don't see how we have the grounds to collect that. Therefore, we can't even ask it.

343
00:53:30.420 --> 00:53:43.280
MTS - A/V: and store it in a volunteer basis. And the text here says that the ASO/AC groups should agree that the following seven elements of diversity should be used as a common starting point.

344
00:53:43.480 --> 00:53:55.830
MTS - A/V: for all diversity considerations. So I read at least two shoulds in that sentence, so I find it difficult to, as a starting point, eliminate any of them. It sounds more like we should add more.

345
00:53:57.490 --> 00:54:21.420
MTS - A/V: 1.2 actually says that you need to identify which are relevant to the to you, for instance, and I did it that way. So of course, I think 7 are part of the diversity, and this is what they they say. And then they say that, okay, you identify which were not relevant to them, and you regularly publish the status of the that. I think it is that way it could be incorporated.

346
00:54:22.670 --> 00:54:24.470
MTS - A/V: We have Rob, and then Christian.

347
00:54:27.090 --> 00:54:30.960
MTS - A/V: Hi, Rob Carolina with, RSAC ISC.

348
00:54:31.100 --> 00:54:32.170
MTS - A/V: F root.

349
00:54:32.710 --> 00:54:38.509
MTS - A/V: My apologies, because I was just sitting here, and it took me a minute to connect the dots.

350
00:54:38.850 --> 00:54:46.689
MTS - A/V: I just realized that I I worked with the workstream to cross community group

351
00:54:46.930 --> 00:54:50.210
MTS - A/V: Implementation Committee that discussed exactly what we're…

352
00:54:50.530 --> 00:54:55.539
MTS - A/V: talking about here, and and that work, I think, predates the

353
00:54:55.730 --> 00:55:01.340
MTS - A/V: the group we're referring to. I mean, our outputs from that group became inputs

354
00:55:01.860 --> 00:55:04.180
MTS - A/V: For this, for this latest group.

355
00:55:05.570 --> 00:55:14.400
MTS - A/V: One thing that I can say with a very high degree of confidence from my time in that work group is it was recognized that

356
00:55:14.900 --> 00:55:19.710
MTS - A/V: These sets of criteria that we're talking about right now are

357
00:55:20.340 --> 00:55:29.400
MTS - A/V: not always equally applicable to every SO and AC in… ICANN

358
00:55:30.060 --> 00:55:36.100
MTS - A/V: and there, for an organization like Rsac, for example.

359
00:55:36.460 --> 00:55:42.399
MTS - A/V: where the… Goal is to produce advice.

360
00:55:42.960 --> 00:55:54.560
MTS - A/V: For the community, but the membership is driven around representatives from a series of identified root server operators.

361
00:55:57.230 --> 00:56:00.950
MTS - A/V: That's not what a lot of these criteria were designed for.

362
00:56:01.140 --> 00:56:06.540
MTS - A/V: It just wasn't. There was. There was an absolute recognition of that in the work stream 2.

363
00:56:07.280 --> 00:56:15.730
MTS - A/V: implementation group. So I would have a high… degree of confidence.

364
00:56:15.910 --> 00:56:25.770
MTS - A/V: that RSAC should be able Very quickly, to look at A number of these points.

365
00:56:26.090 --> 00:56:27.380
MTS - A/V: and say.

366
00:56:27.570 --> 00:56:31.339
MTS - A/V: They simply are not applicable to RSAC.

367
00:56:32.490 --> 00:56:33.610
MTS - A/V: Because.

368
00:56:34.760 --> 00:56:42.780
MTS - A/V: that's not the nature of what RSAC is doing for ICANN and for the internet.

369
00:56:43.170 --> 00:56:48.269
MTS - A/V: That's not who we are, that's not what we are. And as Hans Petter

370
00:56:48.830 --> 00:56:54.490
MTS - A/V: Rightly points out, each of the root server operators sends One.

371
00:56:55.250 --> 00:57:00.200
MTS - A/V: member. I'm sorry. 2 members to Rsac. There's

372
00:57:00.950 --> 00:57:05.339
MTS - A/V: I… we can't even begin the process of figuring out to coordinate

373
00:57:05.690 --> 00:57:19.709
MTS - A/V: which server operator is going to decide how to alter the composition of their delegation. I mean, if we were to hypothesize, you know, if RSAC were the type of organization that had 180 members.

374
00:57:20.310 --> 00:57:26.150
MTS - A/V: You know, popularly elected, or had 200… was a… was a, you know, some sort of a…

375
00:57:26.260 --> 00:57:44.679
MTS - A/V: a parliamentary-type organization with 210 members, and they had to choose a 40-member executive committee, or so… you know, there's all sorts of things that can be done with those types of structures, but that's not what RSAC is. RSAC doesn't represent

376
00:57:45.840 --> 00:57:47.280
MTS - A/V: a community.

377
00:57:48.170 --> 00:57:53.659
MTS - A/V: RSAC is a group comprised of people who have been assembled.

378
00:57:53.920 --> 00:57:56.069
MTS - A/V: To provide advice about.

379
00:57:56.300 --> 00:57:57.800
MTS - A/V: The root server system.

380
00:57:58.900 --> 00:58:01.210
MTS - A/V: That's it. It's a niche role.

381
00:58:01.340 --> 00:58:02.950
MTS - A/V: And…

382
00:58:03.560 --> 00:58:20.899
MTS - A/V: I would applaud Hans Petter for pointing out what should be the obvious as well, which is that I'll go a step further, Hans Petter. It's not just that the privacy laws of some countries limit the ability to ask for or hold this data.

383
00:58:21.250 --> 00:58:26.790
MTS - A/V: the privacy laws of every country I've encountered…

384
00:58:27.790 --> 00:58:30.990
MTS - A/V: with the notable exception of the United States.

385
00:58:32.140 --> 00:58:38.810
MTS - A/V: Limits severely under the GDPR. Of course, this is special category data.

386
00:58:39.000 --> 00:58:49.329
MTS - A/V: which you'll be very familiar with, obviously in Europe. But that model is replicated in so many places now throughout, you know, Latin America and Asia.

387
00:58:49.400 --> 00:59:00.659
MTS - A/V: the number of countries that don't have that protection. The only way to gather that kind of data, and, you know, I'm sure that… I'm sure that… I mean, ICANN have a number of very, very competent

388
00:59:00.810 --> 00:59:06.080
MTS - A/V: Lawyers, I'm sure, who I know from working with them, they're very, very capable.

389
00:59:06.380 --> 00:59:14.770
MTS - A/V: folks, you know, they're going to have to go and request express permission from anyone before anyone discloses the data. So…

390
00:59:15.050 --> 00:59:20.510
MTS - A/V: I, I would hope that we could navigate ourselves pretty quickly.

391
00:59:20.660 --> 00:59:23.359
MTS - A/V: Past this particular point, in terms of…

392
00:59:23.570 --> 00:59:30.580
MTS - A/V: Feeling comfortable and confident that we will be able to write a couple of sentences when the time comes as to why.

393
00:59:31.150 --> 00:59:36.120
MTS - A/V: This is just not the sort of thing we need to be concerned with in terms of assessing our

394
00:59:36.760 --> 00:59:38.290
MTS - A/V: or function.

395
00:59:41.630 --> 00:59:42.410
MTS - A/V: Kirsten.

396
00:59:44.160 --> 00:59:48.120
MTS - A/V: Thank you. I just wanted to provide, because I was able to pull up the.

397
00:59:48.570 --> 00:59:55.629
MTS - A/V: sample survey, that staff, helped draft back in, looks like, 2023.

398
00:59:55.940 --> 01:00:02.460
MTS - A/V: And just to give some, some context as far as the questions that are suggested to be asked, or just

399
01:00:02.620 --> 01:00:08.819
MTS - A/V: available to be asked. The first one, with regards to disability, is about

400
01:00:09.330 --> 01:00:17.709
MTS - A/V: Providing consent to process personal data, and the anonymization of the survey results for if it were to be published.

401
01:00:18.040 --> 01:00:22.190
MTS - A/V: But the questions… Ask.

402
01:00:22.590 --> 01:00:29.590
MTS - A/V: Do you have any physical disabilities or accessibility needs that are relevant to your participation in ICANN? Yes, no, prefer not to say.

403
01:00:30.260 --> 01:00:36.160
MTS - A/V: And if yes, what accessibility needs do you have that are relevant to your participation?

404
01:00:36.360 --> 01:00:40.329
MTS - A/V: And how well are these accessibility needs being met? A scale of 0 to 5.

405
01:00:40.750 --> 01:00:55.160
MTS - A/V: Those are the only questions. So there's no ask of, what is your disability? There's no ask of anything like that. It's just a matter of… it's framed in the fact of, do you have accessibility needs that are not being met? So I just kind of wanted to…

406
01:00:55.450 --> 01:00:57.160
MTS - A/V: Provide that kind of clarification.

407
01:01:02.420 --> 01:01:05.059
MTS - A/V: And also not to say that those questions have to be asked.

408
01:01:05.670 --> 01:01:06.949
MTS - A/V: Or phrased in that way.

409
01:01:19.230 --> 01:01:23.919
MTS - A/V: Our next steps with the work party, or the next steps with what's on screen right now?

410
01:01:24.330 --> 01:01:28.319
MTS - A/V: It would be to incorporate them into our CIP indicators.

411
01:01:30.640 --> 01:01:36.890
MTS - A/V: So you have not just taken the list of 62 and added another, so we know up to 70 plus.

412
01:01:43.390 --> 01:01:48.239
MTS - A/V: I mean, I'm pushing for this because you started saying how the goals should be smart, right?

413
01:01:48.350 --> 01:01:54.559
MTS - A/V: And then that means that we're looking at indicators that we set a goal or a target for.

414
01:01:54.740 --> 01:01:57.749
MTS - A/V: Now, if we measure gender diversity.

415
01:01:58.220 --> 01:02:13.029
MTS - A/V: What's the goal? What's the target? Right? Is the target as Norwegian board again, you by law have to have at least 30% members of the board of either gender. Do we want to, you know, is that the target?

416
01:02:13.280 --> 01:02:20.680
MTS - A/V: for caucus, for instance, right? And that is more important than other criteria. I

417
01:02:21.040 --> 01:02:37.979
MTS - A/V: I wouldn't have to go the way you said, Jeff, that we would have to relax some of the skill sets criteria in order to meet some of the other criteria. I really want to find people that are qualified and that… and still fulfill the diversity criteria. Sometimes that's…

418
01:02:38.210 --> 01:02:42.529
MTS - A/V: Difficult, but, you know, it's so difficult to get,

419
01:02:42.890 --> 01:02:46.800
MTS - A/V: and set the wrong target. So, for these ones.

420
01:02:47.290 --> 01:02:55.430
MTS - A/V: What's the target, right? We measure this, but then some people will say we're doing bad because we don't reach some targets set by others.

421
01:02:56.020 --> 01:02:59.460
MTS - A/V: And then… How do we relate to that?

422
01:03:04.990 --> 01:03:07.030
MTS - A/V: I'll defer to the new chair.

423
01:03:13.660 --> 01:03:14.570
MTS - A/V: Okay.

424
01:03:14.770 --> 01:03:18.649
MTS - A/V: Okay, look like we just discussed this at lunch. This is kind of funny.

425
01:03:19.410 --> 01:03:34.910
MTS - A/V: like Hans Petter, I have this issue in my own organization, where ideally, we would love to be diverse in everything from geography to gender and all the rest of it. The lengths we go to attempting to hire

426
01:03:35.130 --> 01:03:45.969
MTS - A/V: Women, for instance, who are C-coders, who are in the DNS world enough that, you know, we can start from somewhere, has been frustrating in the extreme.

427
01:03:46.080 --> 01:03:49.649
MTS - A/V: So… You know, if you can measure the effort

428
01:03:49.860 --> 01:04:01.149
MTS - A/V: somehow, and just say that you're really trying. That's one thing. But you know, if we were to come up with which bit 30%, if somehow, 30% of my developers tomorrow had to be

429
01:04:01.550 --> 01:04:08.859
MTS - A/V: either men or women, or I don't know how you how you deal with it with the whole, the whole panoply of genders.

430
01:04:09.670 --> 01:04:15.220
MTS - A/V: You know, it just, in practical terms, it just, it gets really, really hard.

431
01:04:15.380 --> 01:04:18.859
MTS - A/V: And it's even harder because there are,

432
01:04:19.380 --> 01:04:27.980
MTS - A/V: you know, this… this gets out of… we all like thinking all problems are solvable by technology. Well, this is… this is kind of a political one, and then I…

433
01:04:28.170 --> 01:04:39.039
MTS - A/V: Guess I'm going to be the first one to throw up there that ICANN is an American corporation based in California. I think the current administration considers requiring this kind of list illegal.

434
01:04:40.140 --> 01:04:41.110
MTS - A/V: literally.

435
01:04:41.320 --> 01:04:46.779
MTS - A/V: So that's why I threw this to to Lehman, because

436
01:04:47.630 --> 01:04:56.010
MTS - A/V: This is something hard that all well-intentioned people are trying to solve, and it's just complicated in the extreme.

437
01:04:59.290 --> 01:05:13.120
MTS - A/V: So I don't know what to say. And then further organizing and quantifying it and documenting it and requiring that it be met in spite of it being so challenging to even define, I just, I'm…

438
01:05:13.750 --> 01:05:19.059
MTS - A/V: You know, this is one of those things that literally makes me thrilled at the end of the week. This is no longer my problem.

439
01:05:29.360 --> 01:05:38.839
MTS - A/V: Thank you. I would just suggest that, at least for this first CIP cycle, that you don't need to set Benchmarks, particularly with regards to diversity.

440
01:05:40.900 --> 01:05:50.479
MTS - A/V: As was noted on the screen that we just moved off of, just conducting the survey enough is… is… Good progress, or showing that it's being made.

441
01:05:50.620 --> 01:05:55.029
MTS - A/V: On rec… for Rec Stream 2, Recommendation 1.4.

442
01:05:55.230 --> 01:06:03.749
MTS - A/V: Says to publish… Diversity criteria, objectives, and strategies for achieving these.

443
01:06:03.970 --> 01:06:05.929
MTS - A/V: on on their official website.

444
01:06:07.110 --> 01:06:15.449
MTS - A/V: That is obviously up to RSAC what it thinks its diversity criteria, objectives, and our strategies should be.

445
01:06:15.610 --> 01:06:32.710
MTS - A/V: What is even relevant, what is… what is practical, or not. So, I would just suggest, at least for this first cycle, and if it were to incorporate into CIP, to keep it simple, and not try to… to set expectations that are unreasonable.

446
01:06:32.970 --> 01:06:43.829
MTS - A/V: Also, recognizing that it is a very hot topic issue and a sensitive issue, particularly when it comes to publishing that information. So just bearing all that in mind.

447
01:07:00.310 --> 01:07:01.180
MTS - A/V: Frederick?

448
01:07:02.140 --> 01:07:08.920
MTS - A/V: Fredrik Netnod, I wrote, what is the timeframe here? When does this document need to be finished, approved, etc?

449
01:07:09.490 --> 01:07:16.619
MTS - A/V: And are we the ones approving it, or should someone else approve it? Are you asking for the… are the initial indicators for the CIP?

450
01:07:16.870 --> 01:07:17.960
MTS - A/V: Exactly. Yeah.

451
01:07:18.480 --> 01:07:23.230
MTS - A/V: We have a… I think our goal is within the next few months.

452
01:07:28.870 --> 01:07:40.909
MTS - A/V: Yeah, I think. 3 months to 4 months we should be done with, maybe before the next ICANN meeting, or by the next ICANN meeting we should have some some

453
01:07:41.030 --> 01:07:45.830
MTS - A/V: Agreement on… on… What are those, right?

454
01:07:45.910 --> 01:07:58.949
MTS - A/V: As I said, a second part would be how… what would be the steps after, what are the implementation plans for RSAC, because RSAC might choose to have a subset of those

455
01:07:58.950 --> 01:08:17.999
MTS - A/V: right and say that, okay for a pilot, we would go for that. But that's II think, in the in the working party document. It's not mentioned that one. So our remit as of now would just be to to fix those and identify those criteria and indicators.

456
01:08:27.040 --> 01:08:28.680
MTS - A/V: Christian and then Kent?

457
01:08:30.939 --> 01:08:42.480
MTS - A/V: I'm sorry, I think that's an old hand. But I would just say that the… yes, as far as agreement on criteria indicators and how those are evaluated, would be great.

458
01:08:42.910 --> 01:08:50.729
MTS - A/V: As well, if that includes consolidating some or reducing some, updating them, that sort of thing. So, I think that that's a worthwhile goal.

459
01:08:53.680 --> 01:08:58.349
MTS - A/V: This is Kent. Just 2 suggestions for the the work party.

460
01:08:58.600 --> 01:09:00.860
MTS - A/V: I know it's been a while since I've looked at this document.

461
01:09:01.330 --> 01:09:06.090
MTS - A/V: If we were to just look at the criteria without any indicators, just the criteria.

462
01:09:06.660 --> 01:09:09.800
MTS - A/V: Just are we still? Is this still a good list?

463
01:09:10.029 --> 01:09:14.759
MTS - A/V: Are there any things that are missing? Any things that can be thrown away or combined?

464
01:09:15.000 --> 01:09:17.810
MTS - A/V: And then… After that.

465
01:09:18.040 --> 01:09:21.209
MTS - A/V: Let's pick some of the easy ones. Easy.

466
01:09:22.000 --> 01:09:27.399
MTS - A/V: principal criteria pairs that have an existing set of of indicators.

467
01:09:27.750 --> 01:09:32.289
MTS - A/V: Look through that, and, you know, let's… let's try… let's try the easy ones first.

468
01:09:32.720 --> 01:09:35.349
MTS - A/V: And see how it goes, and then…

469
01:09:38.390 --> 01:09:41.919
MTS - A/V: Those can be our initial ones to go through the first… first round, but

470
01:09:42.189 --> 01:09:44.750
MTS - A/V: I think it'll give us a lot of experience.

471
01:09:44.850 --> 01:09:47.350
MTS - A/V: Ability to do the slightly harder ones.

472
01:09:50.069 --> 01:09:51.450
MTS - A/V: To follow that, yeah.

473
01:09:52.740 --> 01:10:00.940
MTS - A/V: Frederick not here again. I was about to say the same thing as Ken did, with the addition or change, that a lot of these indicators are…

474
01:10:01.110 --> 01:10:05.910
MTS - A/V: quantitative, right? I think we should have fewer indicators, and they should be qualitative.

475
01:10:06.120 --> 01:10:19.990
MTS - A/V: So the goal doesn't have to be a hard number, because then you usually get into that the number should be higher or lower, it should be zero or a gazillion, right? But usually, we need some qualitative good enough measure.

476
01:10:20.120 --> 01:10:22.099
MTS - A/V: Not an absolute number.

477
01:10:22.660 --> 01:10:27.819
MTS - A/V: And then I was about to say something else. I forgot. I will take the mic again if I remember.

478
01:10:33.110 --> 01:10:42.489
MTS - A/V: So… I'm not sure I understand how you can measure something without numbers, but we can have that discussion.

479
01:10:42.650 --> 01:10:50.620
MTS - A/V: Also, we should, and this is a comment from the SOAC chair's meeting yesterday.

480
01:10:50.800 --> 01:11:07.330
MTS - A/V: ICANN has been very good at measuring effort. We need to measure outcomes. So anything that's effort-based, we could basically strike from the list and look for how do we measure outcome. So number of meetings or number of reports, that's kind of effort, but

481
01:11:08.160 --> 01:11:12.829
MTS - A/V: It's the outcome. And how on earth do we measure that? That's the really difficult part.

482
01:11:18.200 --> 01:11:26.919
MTS - A/V: So yeah, that's pretty much the time that we have here. I'll pass it off to Naveed for some closing remarks, and then I'll give an update on our next work party meeting.

483
01:11:28.100 --> 01:11:42.759
MTS - A/V: Yeah, I thank everyone for the participation and looking forward to your comments in the document directly. So all those some of those that, for example, you you in said about quality versus quantity.

484
01:11:42.760 --> 01:11:49.660
MTS - A/V: Please put those as as a starting point in the document, and we can

485
01:11:49.660 --> 01:11:58.019
MTS - A/V: continue our discussion and looking forward to to the next call for for the working party. Thank you everyone for

486
01:11:58.090 --> 01:12:12.290
MTS - A/V: Participation, thank you. And we are highlighting for our next call right now, we are looking at November 2nd, Monday, November 2nd. I believe that will be at 16 UTC with the time zone shift?

487
01:12:12.850 --> 01:12:15.589
MTS - A/V: So, keep an eye out for a calendar invite.

488
01:12:16.210 --> 01:12:19.320
MTS - A/V: And, Jeff, want to close this out?

489
01:12:20.350 --> 01:12:26.490
MTS - A/V: Thank you, Naveed. Appreciate the effort very much, Daniel, as always for the magic. Thanks. Please stop the recording.