An interview on CGM accuracy and standardization with Guido Freckmann, MD, Medical Director of the Institute for Diabetes Technology Research at Ulm University in Germany.
An interview on CGM accuracy and standardization with Guido Freckmann, MD, Medical Director of the Institute for Diabetes Technology Research at Ulm University in Germany.
Welcome to Diabetes Technology Report. This is the podcast devoted to diabetes technology. I'm Dr David Klonoff. I'm an endocrinologist at Mills Peninsula Medical Center in San Mateo in UCSF. I'm going to introduce our other co-host, Dr David Kerr, who will introduce our guest today.
David Kerr:Thanks, david, and hello to everyone listening today. I'm David Kerr. I'm a diabetes researcher and based in Santa Barbara, california. Today's very special guest is Guido Freckman, who many of you know is really a world authority on glucose monitoring in all of its shapes and forms. Welcome, guido. It's great to see you and hear from you, rather Just to begin for our listeners to really to set the scene. How did you get interested in diabetes technology, glucose monitoring? Where did that come from?
Guido Freckmann:Yes, David, Thank you very much for your kind introduction. To introduce myself, I'm Guido Freckman from the Institute of Diabetes Technology in Old Germany. It was a bit accidental that I came to glucose monitoring. I came to this institute in 1999, starting with research on algorithms Today you would say for AID systems. From these studies we gathered a lot of BGM data. After a couple of years of these studies, we started with the evaluation of BGM data and then recognized that there became a standard available. We started performing studies according to the standard.
David Kerr:What are you working on at the moment in this area? What's your passion today?
Guido Freckmann:Yes, after working more than 10 years, we did lots of evaluation studies with BGM. In parallel we did many studies for the development of CGM or for CGMs under development Currently. We performed both. In the last years we tested a couple of CGMs which are currently under development. We also performed BGM studies. Since 1999, I'm chairing a working group on CGM to bring something that's already available in BGM an international standard also to the CGM scene. I'm very proud that David Klonoff is also one of the members of this working group. We are looking on study procedures on metrics and under gaps. We are also looking on study procedures on metrics and under gaps that's left by the current guidelines and performed some publications and some work in the last years. Our goal is to support the development of an international standard to make CGM studies more comparable.
David Klonoff:Guido, why do you feel that it's important to make CGM standards comparable?
Guido Freckmann:Yeah, I think there are currently some really good CGMs available, but the whole study procedures are not defined, that it's difficult to compare the results of different studies and especially now we see, especially here in Europe, many devices coming from Asia now and most of them are proposing a very good mark. It's very difficult to distinguish. Is this really the accuracy you will have in daily life, or what is better, what is worse? And it would be very helpful if we have a standardized procedure that you can better compare different devices and see is it good for adjunctive or non-adjunctive use.
David Klonoff:Guido, do you think that MARD is the best metric for defining the performance of a CGM, and could you also comment on the metric that you developed, cgdiva? Yeah, so.
Guido Freckmann:I would say MARD is the most used metric for CGM because it's only one parameter and it's easy to show this.
David Kerr:Just for our listeners. Can you explain, mard, just so that everyone's clear what that means?
Guido Freckmann:So the MARD is a mean, relative, absolute difference and it's a comparison between the CGM data and the comparator data, which is typically a BGM or a BGM lab device, and you take the average of all the differences you measure in the study and it's really very dependent on study design, on frequency of measurements and so on, and so it's averaging a lot and not really showing the difference in the details. And that was a reason for us to develop the CGDiva, which is based on the ICGM criteria of the FDA and we try to visualize the data, to show the median and the distribution and to give, on a visual base, more information than the MARD has or the MARD provides Guido.
David Klonoff:one more question if a device has a high CGDiva, meaning that there's a wide distribution, how would you use a device like that differently than another device which has a narrow distribution?
Guido Freckmann:Yeah, so I would first look. We have proposed two plots, so an average plot including all the data and then a plot with the single devices. And I would look how are the single devices are working? Is there a high distribution between the devices? Is there a high difference between single devices? And if the or is there a systematic bias which is reproducible and you can correct for? Or is there a high noise? So, looking on all this data, you can distinguish between the devices and say, okay, it's good for this or probably not so good for that goal.
David Kerr:Guido, just taking that rather technical description there, looking at it from a point of view of people with diabetes. Why is this important and does it matter in different situations? This accuracy, why does it really matter?
Guido Freckmann:no-transcript. I think it matters, especially if the and that's another proposal we made for the study procedures if there is a higher rate of change and you should be one before a hypo and if you have a long time delay, the warning is probably late. And these are situations where the study procedure is important. And for the MART in the CGDV you can see for the lower values and the higher values is there the same bias or is there probably a different bias that hypo's are over or underestimated and that can really have impact for the diabetic and for the hypo warning and other things in his daily life.
David Kerr:So this is really important and probably it's going to affect the decision making about which technology is the most appropriate for people with diabetes. That's where this is going.
David Klonoff:Guido, what would you say is the effect of safety? If you have either a high MART or if you have a wide CGDV, will this affect safety?
Guido Freckmann:So it's very interesting to look especially on the single sensors. If you have a high distribution of all the sensors, I think this affects safety. If most of the sensors are looking good but only one or two are deviating, it could affect safety for this one or two and it's important to detect them. And yeah, I think it depends on the degree of differences in this plot if it affects safety or not, but it can affect.
David Klonoff:Are you seeing products coming onto the market in Europe where there's no published data about their accuracy?
Guido Freckmann:Yeah, most of the products coming to the market have their own data and the question is how this data are achieved. Is there how many rates of change, distribution of values and so on? So we tested last year a device that proposed to have an MRD below 10 and with a factory calibrated mode. It wasn't about double the MRD and with one day calibration it was still 15. So I think it's very important to have comparable procedures to really see to get comparable values, output values out.
David Kerr:Yeah, so it seems that's very reassuring for people with diabetes and clinicians that this accuracy question needs to be constantly thought about. I'm sure many of our listeners would like to know where do you see non-invasive devices at the moment? Where are they at the moment in terms of accuracy? Is this really going to happen soon or is this still a pie in the sky?
Guido Freckmann:So my insight into non-invasive devices is limited, but I know some devices which can already measure I'm not sure in all devices around there and, interestingly, we bought two watches who said they can measure sugar from Asia, and my colleagues wear these watches two days and they show on each day the same curve. And on the third day we gave it to a banana and the banana showed the same curve, like my colleague. So these watches are already available. We got them from 50 to 200 euros and I think in future we will have working non-invasive devices. But I think, as I know, the current non-invasive devices are there where CGM was about 10 years ago or a little bit more, and they are developing and they will come, but I think we need some more time.
David Klonoff:Peter, what advice would you give to a scientist who wants to develop a new glucose monitor?
Guido Freckmann:Yeah, I think many people, especially in the development of non-invasive devices, are very optimistic and they should be, in some case, realistic and looking. What are the first steps? You need to go step by step and you will not have with the first device, and that's the same with CGM sensors. You will need some years to improve your algorithms and your device to be in the status where the current CGMs or VCGMs are.
David Klonoff:Is there anything else that you'd like to tell the audience about your experiences or your recommendations?
Guido Freckmann:Yeah, currently. I already told that we submitted a procedure. We think is very important to use a comparable procedure for the studies and we are currently preparing a study for this and currently looking for funding, which is not very easy, and I hope we can get the funding that we can perform a study with the currently well-known devices to set something like a benchmark with the study. Where is it if we use more parameters like CGDVA and not only MRD, with a really defined study procedure?
David Kerr:Just a philosophical question. Is it ever going to be the case that the difference is going to be zero, or is it a? What's the holy grail of accuracy for glucose monitoring? Do we know what that is, or is it just pull it?
Guido Freckmann:there. That's a very good question. If you measure capillary or venous you have a difference and we have seen in health C's in an OGTT post-prandial up to 30% difference between these two compartments. So it's a bit difficult in the body. It depends on the place you measure. You will have differences and therefore the accuracy need to be for a degree that is, I would say, within 10 to 20% to make no larger failures. But it's very difficult to. You can measure it in a lab device to a high degree of accuracy, but on the way to the measurement are already differences which you cannot get rid of.
David Klonoff:Guido, thank you for speaking with us today. You're definitely a world leader in glucose monitoring and I hope people listen to what you had to say so to the audience. Thank you for listening to the diabetes technology report. We're available on Spotify and the Apple store. On behalf of David Kerr and myself, Guido Fragman, thank you and we will catch up with you at the next podcast. Bye, everybody. Thank you very much.