About : standard furniture kathy ireland home collections
Title : standard furniture kathy ireland home collections
standard furniture kathy ireland home collections
welcome. i'm eric dishman general manager of health and life sciences of intel i'm happy to host everybody here in santa clara. the headquarters for intel worldwide. i was hoping you could wear your bunny suits and see what is it
is like to work at intel. so, if you think about what is happening in some of the fabs around us, we are working on the next generation of microprocesses, so small they'll go into flexible electronics we can wear all the time up to the high performance computing
systems that will, today's precision medicine analytics are already taxing. as we think about the scale of precision medicine going forward, the compute challenges from the wearables to the back end are enormous. i thought i'd share a quick
story by way of introduction. it's about 13 years ago my intel team, we had been studying the needs of people with alzheimer's and we were able to find from korea, the very first shorted of small phone, that had gps in it, and we had built some sensors that went into the homes of
people. our goal to help people with alzheimer's to remain-- maintain daily independence. so we had sensors to help track activities of daily living. coach this woman, i remember from the field work in particular, named barbara
struggling to try to make tea for us when we arrived. our system was trying to use intelligence and sensors to be able to help her do tea, make tea, on her own and maintain some independence while at the same time, collecting a whole wide range of datasets.
this was almost impossible 12 years ago. it was very expensive. wearable technologies had not taken off. and we were faced as we did this, with an enormous challenge because we had all these data streams and we were like, well,
one goal was to use these data streams to help people maintain quality of life and the other was to have sinusitises into things we had never understood before about the emergence of cognitive decline. it was too hard know with all that personal data, in-home
data, what was going to be medically meaningful. fast forward 13 years, i had my team, some of them here today, we have a particular cardiovascular a wearable pilot we are doing with 500 people right now and i said, how is it going?
they checked the servers last night and we are pulling the 1.5-2.5 million data elements tonight for just 500 people. and the compute power we needed 13 years ago was in awarable in a few places and sensors around the environment. the same question remains.
that data, while powerful in giving people in bites their own lives, can it give researchers and medical experts and clinicians insights? and we turn it from insight to knowledge to do something with? how do we make meaningful use of personal and mobile
technologies? that is going to be a huge thing today. so i will shut up and dot next step of welcoming dr.francis collins. dr.collins needs no introduction. he has been our fearless
champion for the president's call out for precision medicine initiative and it's a great honor to have you here for the work group and especially to welcome dr.collins. [ applause ] >> thank you eric, and good morning to you.
i really want to thank eric and his team here for helping us organize this and hosting this in such a nice place and good environment for the conversation we are going to have and particularly i want to mention thanks to april hanks, eric's assistant who i know has done a
lot to make this happen. i should also thank our working group co-chairs who have been prettying their lives on hold for the last four months or so. that being of course rick and bray and cathy. and all of you on the working group who have pretty much also
agreed to take this as a high priority in the midst of every other thing you might have been planning to do this summer. i think we are on a good path to deliver what is expected, which is by september, a plan for which this cohort is really going to look like.
and these workshops have been critical for getting that kind of input we needed in order to get there. cathy will come up in a minute and lay out the context of how we have gotten to this point in terms of what the precision medicine initiative aims to do
and what we have accomplished so far. particularly i want to thank for today's conversation and tomorrow, about mobile technologies from nih bill riley, roderick ped grew, allen guttmacher who helped us to try to put together the program.
i will tell you that i'm really excited about being here because this has been a personal interest for me for several years. the idea of using m-health to advance the cause of how do you maintain health and how do you manage chronic illness?
i have been geeking out about this for a long enough to be pretty jazzed about what we are going to hear today. i was volunteer once upon a time in a live course test of their ekg transmission capabilities, which was kind of found see what happened.
and certainly have spoken repeatedly at the m-health conference which is have turned into big gatherings every year in the dc area and yet, it's also clear from tracking how this is going on that there are lots ask loss of really exciting gadgets-- i'm sure many are
wearing them. i have two on my wrist at the moment. but the questions that remain-- largely unanswered, at least in my perspective, are exactly how we can implement these technologies in a way that you can prove outcomes are
benefited. and that it is not just a lot of cool technologies and gadgets but we are actually able to generate evidence about how this changes health behaviors and improves outcomes and management of chronic disease and so on. it seems to me that this
precision medicine cohort of a million or more people, i'm still sticking to that, or more, because i though is more-- will provide the kind of real world laboratory to enable a great deal of information to be gathered about which of these various approaches actually do
turn out to be useful in terms of improving health outcomes? a lot of possibilities here. we all know there is 7 billion cell phone subscriptions worldwide now. and gathering all kinds of data that should be potentially relevant to health.
and we have seen applications in many different situations that seem to be showing some real benefits, parkinson's disease, post-heart attacks, smoking cessation, clinical trial monitoring and nih is investing in various research projects to look at m-health technologies in
obesity and diabetes and substance abuse, heart disease, psychiatric illness, depression and many others. and so a evaluation is getting there. but still, i would love to see that trajectory sped up and that is what i'm hoping we could
imagine here in the next day and a half might be possible with the way in which the technologies are delivering so much potential and the opportunity in a situation like the pmi, to be able to test those out and see what they could really accomplish.
so, i guess my hope and my charge to the group is, we role up our sleeves here and listen to fantastic presentations. we have discussions, on the panel are supposed to be highly interactive and we organized the agenda in that fashion. so this is not the sort of
meeting where you just come and do your e-mail. this is a meeting where we hope there is lots and lots of interaction among you who have turned up and the panels that have been carefully chosen. this will all feed into, this effort, which the working group
has been charged with, to deliver a plan in september that can be put forward to the advisory committee to the nih director and if accepted, put into place. i can tell you we did have a meeting about the cohort study with the president of the united
states just a couple of weeks ago. and he is intensely interested in what is happening here and intensely supportive of the goals and wants to be kept apprized of all the details emerging as part of this. you can't ask for a more
connected kind of leader of the free world in terms of what this particular effort is turning out to represent. and that is really quite gratifying to see that kind of leadership. i think in the u.s., the support for this in the congress of the
united states is broad and strong and completely bipartisan. so we have a lot of support. we just have to make the most of it here and turn this into eye policeman that is going to change the who recalled and that's our goal.
so without further adieu to celt the context a little bit more clearly, i'm going to invite kathy hudson, deputy director at nih and co-chair of this working group who is fresh back from riding her bicycle across the entire state of iowa to tell you more about context and many
thanks to kathy who put a huge amount of time and effort into getting us to this point. dr.hudson? >> so good morning, everybody. it's really exciting to be here at our final precision medicine working group meeting. we have a significant challenge
ahead in the next several weeks to put together this report for the advisory committee to the director. but i wanted to make sure, since for some people, this is your first workshop with the precision medicine initiative, i want to make sure that we are on
the same page as we get started. so the challenge is that precision medicine initiative is intended to address our multi-fold. first we all know that too often either we or our family members or somebody that we love end up in a situation where they are
ill and there is either no diagnosis or no options for treatment; and we end up at the end of a road without a happy outcome. oftentimes, the options that are available fail to address the individual's genetics, behavior, background, environmental
exposures and the like. also, research can sometimes be too slow to be able to address many of these problems, too expensive, underpowered, and again don't really address real-world situations. so we see people intermittently in clinical trials or in other
research studies and being able to track people over a extended period of time in their real free-range environments offers tremendous power. one of the key values that has really undergirded all the planning efforts with the having a new relationship with
the participants within the initiative so they are not research subjects. they are not patients, but rather they are our partners and active participants in all aspects of the precision medicine initiative. and then finally, an additional
motivation for the precision medicine initiative is that too often it takes too long for new knowledge to be translated into clinical practice where it can actually benefit individuals and families. so, the president of the united states in the state of the
union, in january of this year, launched the precision medicine initiative and i'm hoping if i click something, this video will start. and this is when he announced the beginning of the initiative. to give all of us access to the information we need to keep
ourselves and our families healthy. we can do this. >> so you'll notice that everybody in the chamber, both republicans and democrats, all rise up in support of this precision medicine initiative which nobody actually knew what
it was and which we have been trying to define. [ laughs ] interestingly, in this age of electronic monitoring, there was polling going on in the state of the union and it was that sentence the single sentence that was most favorably received
by people watching the state of the union address. so there is enthusiasm out maybe for the word precision, maybe for the word medicine, or maybe for what is very clear, the president's own enthusiasm about this initiative. so the time is right for us to
launch this project. i don't know why that is still it shouldn't have been there. so the time is right for this initiative because of advances in all sorts of technologies that make this feasible both from a cost perspective but also from a technological
perspective. so some statistics there about the cost of sequencing, about the increase utilization of smartphones, about emr adoption and other kinds of electronic health information also skyrocketing. so, this is the time to be able
to harness all of these technologies together for the initiative. so, the vision for the precision medicine initiative is to build a broad research program to encourage creative approaches to precision medicine, to test them rig usually and ultimately to
build the evidence base needed to guide clinical practice. and there are two research components of the precision medicine initiative, one in oncology which we are not going to talk about today. and one this national research cohort of a million or more
americans which is the purpose of this workshop and the ones that preceded it. so the goal of the research cohort is to enroll a million or more u.s. volunteers and we have sort of at the ready, numerous existing cohorts that we could utilize to help build this new
cohort. we also want to make sure that we are reaching out to underrepresented and underserved communities to make sure that they have an opportunity to participate and we also want to be able to enable anyone in america to be able to say, i
want to contribute my information in order to advance science and advance precision we need to be able to have a multiple roots of being able to in violate people to participate in this cohort. as i mentioned participants are going to be central evolved in
the design and implementation and governance. and be able to share their genomic information, behavioral information, biological samples, and electronic and other digital health information. we want to make sure that all participants are voluntarily and
actively and enthusiastically participating in this study and making choices about how they participate in this study and what information they received back and i'll talk more about that. and the study will be a new model for engaged partnership
with our participants and also with open responsible data sharing, not only data sharing within the scientific community but also data sharing with the participants themselves. so the cohort has a number of possible uses which are listed here including being able to
provide data to subclassified disease, to be able to test existing pharmacogenetic variants in a broader array of populations and looking and seeing how those variants indicate either adverse reactions or responsiveness to medications, how that works on a
population scale and being able to identify new pharmacogenetic variants that would influence decision-making about what kinds of medicines individuals take. there is a number of possible uses for research cohort and it will be important for us as we build this cohort, to have very
clearly in mind, some research questions that can be answered in the near term and the long-term so we can define very specifically, the data elements that we need to collect. so in launching this study, we know already that there is strong support in the congress
as francis mentioned, extraordinary enthusiasm in the white house with the president of the united states and strong support in average americans. we managed to do a survey earlier this summer of 2,601 folks asking them how they would feel about the creation of a
national research cohort like this. there was very strong support of the idea of about creating a 79% saying this should be built. there was very little variation across racial and ethnic groups in remembers it of that strong high level of support.
when asked, would they participate in such a study after having that described and defined for them, more than half said they would indeed participate such a study with consistently majority support and interest and participating across all groups, although
higher interestingly among hispanics, increased interest in participating with higher educational levels and lower support with increasing age. that is something we'll have to be mindful of to make sure we get everyone involved and engaged in the cohort
so as francis mentioned, after the president's state of the union mention and then another event at the white house about 10 days later, he put together this working group which i had the pleasure and honor of co-chairing with rick and bray over the last many months.
the charge of our group is to design the plan, lay out the blueprint and to do that, we have sought broadly to get input from experts and people with special expertise and experiences that we can draw upon to help build this cohort. and part of that outreach effort
has concluded workshops such as these so we had three previous workshops, one on what would be the scientific opportunities that we could explore with the cohort, one on digital data and how would we build the data architecture for the pmi cohort, one on participant engagement
and health equity, basically talking about how do we get people engaged in the cohort and maintain that engagement over long period of time making sure that we have a very inclusive research population so, that brings us to today and today's workshop and after
today's workshop, we will be very busy working on our report that we will deliver to the advisory committee of the director in mid-september for their consideration and then on to dr.collins for his final decision. we have had a great partnership
between the working group and staff and leaders at the nih so that all of the nih leadership has been paying close attention to the deliberations in these workshops and to the working group so that we are ready when the report is delivered to us, to be able to translate that
into funding opportunities and initiatives that will launch this cohort very quickly. the members of the working group are all seated up here in the front couple of tables and they are all listed here. it is an extraordinarily diverse working group in terms of areas
of expertise and perspective that have been brought to bear. it has been a tremendously productive and engaging group to work with. so, up until this point, we have given a lot of attention to making sure that the cohort is inclusive.
but not necessarily statistically representative of the u.s. population. and that is an important point. it's important both for how we go about recruiting and sort of the general sampling strategy that we will undertake. we want to see where possible,
that we can leverage existing cohorts. we want to make sure we include the underrepresented and underserved possibly by working through federally-qualified health centers. that's a tremendously exciting opportunity with the majority of
folks receiving care in federally-qualified health centers being at or under the poverty line. we want to be able to include people of all ages from the youngest to the oldest and we want to make sure that we enable participation from active duty
and retired veterans and we have been working very closely with our colleagues from the veteran's administration who have the million veterans program already underway. we are learning from their experiences and hope to be able to combine these efforts and
also with our colleagues at the department of defense. and of course we will attain consent repeatedly throughout the life of the cohort for participants because as we move forward, we will adopt new technologies and adopt new research questions and we will
want all participants to be engaged in that. so we have talked also about-- oops, i skipped forward. that's all right. we also talked about data and data management and balancing the need to broadly share information but also be able to
secure people's privacy and make sure that information about individual participants is not used in way that is harmful or hurtful to them. and we have worked very hard on the question of what information is returned to participants. certainly participants are
interested or at least some participants will be interested in knowing research is done with the cohort and what was learned from that and how was it utilized and that is one category of information. there is also information about individual participants that we
have heard through our survey and through other means that people would be interested in getting back. so we need to figure out how to do that in a responsible and appropriate way. so we have a number of challenges in all of those
areas. and today's workshop is really focused on how we can utilize mobile health technologies in order-- in the cohort, in order to collect new kinds of data and new and different ways. we want to explore the benefits and barriers of using mobile
technologies in the cohort. we want to talk about what would be the ideal desired functionality of such technologies. we want to talk about both the technical challenges but also the social challenges in utilizing these mobile health
devices. and then we want to talk about specific examples where we could employ technologies within the cohort and really test their capabilities. so that is the broad goal for our workshop today, is to explore these technologies and
how we can utilize them potentially within the cohort. i'm particularly excited about this workshop. this is not an area i have much past exposure and experience with, so i'm really looking forward to learning from all of the wonderful panelists and
speakers that we have lined up a couple of housekeeping issues before i introduce our first speaker. so this workshop, like all of the workshops that we have held for precision medicine initiative, is available on webcast.
so you can watch this-- greetings to all the people who are watching us right now. also, please if you tweet, please join us #pmi network so please join us on twitter to have conversations within the room and with our fellow friends who are watching on the web.
i feel like tweeting and meet suggests like passing notes or having a separate conversation. it's quite entertaining. so, with that, i will introduce our first speaker jessica mega. the head of clinical science and the baseline study at google life sciences.
as a faculty member at harvard medical school, and a senior investigator with the time study group and a cardiologist at brigham women's hospital, dr.mega left large international randomized trials evaluating novel cardiovascular therapies.
she also directed the time study groups genetics program. she is widely published and has received many, many awards for her work. we are pleased to have her here with us today to talk about mobile and related technologies and their potential for
advancing precision medicine. >> thank you all for the invitation to be here, and without sounding too hokie, i'm also happy to stand up in support of the precision so what i want to do today is to reflect at a high level in advancing the mission that we
have today. speaking of technologies. i'm not sure the click ser working but i'm happy to just say, next slide. so what do we mean by mobile? and that is what i want to spend the next few minutes on. so, if we think about mobile, it
is an adjective. and if you look at the definition, it means of or relating to cellular phones. handheld computer and similar and this is not news to this audience that the original cell phone came out in 1973. it was nine inches by five
inches. you could talk for about 30 minutes and it charged for 10 hours. if that was the case, then it wouldn't be very helpful in the discussions that we have today. but this was a major breakthrough at the time.
and if we look over the decades, what we can see is devices that are becoming more miniaturized, we have devices that are truly mobile and the battery life is getting better and better. and it is because of this we are able to think about new ways to data capture and i hope there is
a lively discussion on this so where are we is in 2015? are these devices the type of devices that people have ready access to? and the short answer is, yes. so 90% of americans own a cell phone. 64% of adults own a smartphone
and if you look at american teens, it is over 70%. so these numbers are only going to be rising. what types of things can be done? this is where much of the discussion today is routed through these types of mobile
devises we can gather patient management information, diagnostics, information management, health record maintenance, no longer do patients need to go or participants need to go to the doctor to get this information. there can be references,
medical, education and understanding of what happens to someone's nutrition and wellness outside of the confines of an office. and the scale and utility of smartphones in addition to this dramatic shift in health care catalyzed bio-sensing wearables.
we heard in the introduction that many people are wearing awarable. i was able to check my heart rate before i gave this talk. we'll see how it goes. i'm starting to feel pretty comfortable. you seem like a nice group.
these are the types of things we have access to every day. and what can you see along the x axis is the many different spaces where these biosensors are infiltrating. anywhere from movement to heart rate, which i think a lot of us think about, but thinking about
oxygen monitoring, muscle function, eye function. so this is where we are going. and the other thing a shift towards these devices just being part of a health and wellness profile to something that can help us manage patients who have more end-stage types of
diseases. so, device that is are measuring oxygen saturation, device that is are going to be measuring things like albonic skin response. these are things that are going to be providing signals that will hopefully give us very
useful early data to predict before a certain disease state happens. but i want us to pause for a member and reflect even more broadly. so we talked about mobile as the concept of mobile devices and mobile phones.
but if you look at the second definition of mobile, what it means is able to move or be moved freely or easily. so where might we be going with these technologies? one area that illustrates this exceedingly well is to look at diabetes management.
there are 382 million people around the world who have diabetes. 79 million adults in the united states have a pre-die-- diabetes state and patients with the diabetes need to check their blood sugar somewhere around four times a day.
we than typically, this means that using a lancit and deriving data. we have many of the physicians in the room have patients who come in with nicely curated notebooks. but this is where we are today. but the hope is to take mobile
technologies beyond the cell phone and miniaturize them in a way that we can work continuously getting information. and this morning, andy conrad who heads google life sciences, happened to have one of these contact lenses in his pocket.
i'll put right here. this smart contact lense is a monitoring device that hopefully will be to continuous noninvasive monitoring. what is really interesting about it, if you take a look, there is say very small sensor. the power device is all
internalized. and the chip requirements are starting to get more and more efficient as we heard in the introduction today. so, we have wireless power. we have biocompatible substrates and there is vast wireless read out and this is just the tip of
the iceberg. this is just one example of where these types of mobile devices are going. in fact, if you think and shrink down to a smaller level, we talk about nanodiagnostics, so noninvasive methods where somebody may be administering
nanoparticles these circulate within the body and this is much smaller than the contact lense. we are talking about microscopic particles and then it shares the information with doctors. the hope would be these kind of mobile, using the word, very broad now, but these mobile
types of devices may be able to detect circulating tumor cells before a mass is detected. but now i want to get even smaller and then pop out to think, what if mobile isn't something that you even need to wear? or put on or inject?
is the future of a wearable, a wearable? not a wearable? but what do i mean by this concept? we have devices that detect whether we are going to have a fire. there are codes we have to have
these devices. there are environmental detectors now looking at our environment. smart ways of managing your home heating, your home cooling. these types of devices are thinking for us, detecting signals and they are trying to
make very smart and active decisions. and now i'm talking in a bit more of a hypothetical space but again i think these are the types of sensors that are already being developed of the so let's think about arrhythmia management.
i'm a cardiologist sensitive to patients who die of sudden cardiac death. but what if there were sensors within your home that detected when your heart went into arrhythmia? they alerted 911 and emergency services could be delivered?
so that is one way of thinking about sensors more broadly. mobile but not smug may need to wear. there are also people interested in seizure activity looking at the brain activity or looking at motion detection. another area where these
noninvasive types of sensors could be considered. hypothetical but something people are starting to think about. and then let's take it a step further and think about the environment in which we are embedded.
i know there is a discussion today around trying to measure the environment that we are in. let's say that the you had your personal health data and you went to the hotel, many stayed locally so welcome to california. let's say you went to the ho
dispel you also provided your health information and said, on this chip it shows that i have a peanut allergy. and as it turns out, we know that peanut allergies kill many people every year. i saw on the back a table of peanut cookies so for who are
sensitive, it's being raised rye now. be aware. but let's say you checked into a hotel. we knew you had a peanut allergy. you were able to detect the environment looking for these
allergens and there was a closed feedback loop. so there was a hvac system able to extract the allergens? this is the way the environment is tarting to be mobile and is starting to detect all of these signals. something we hope to strive for.
if you look at the nih mobile page, i found it really interesting. the information quite comprehensive. and the hope is quite broad. the hope is that mobile technologies could and again they use the word potential and
could, which i think is very appropriate. but this could change when, where and how health care is provided. it can ensure that important social and behavioral and environmental data are used to understand the determinants of
health and i think really important is something that kathy mentioned this morning, the hope is to truly improve health outcomes. not to just gather a lot of information which is kind of fun but it is to really do something different for participants and
for patients. and so what is the own us on us? what are we going to need to consider? there is going to be an exponential update and everyone in the room knows this. you had working groups talking about the genomic and molecular
markers and the organ itself and then the exposures. but we are going to be very careful with this data and be very thoughtful. so we know that there is certain known signals. i showed you the example of ventricular tachycardia
arrhythmia on the top of the slide. we know that atrial fibrillation exists. there are device that is can help detect it. if a patient has this condition, we can use antithrombotic agents early and prevent strokes.
here i would argue are known they are great to go after and they are actionable and something we can do about it. but there is also going to be a lot of unknown signals. and how do we thoughtfully think about these signals that may have a more ser cool us path and
may really hold the true gem we are looking for? and i think that is where initiatives like the precision medicine initiative come into play. because you're hearing these concepts of having over a million patients and
participants that are engaged in providing all of these human signals and then people like you in the room and people across the country, who are going to be looking that the data, be help tease out what is a true signal and what is actionable. at google life sciences we have
been engaging in work around the baseline study and started a pilot that has been publicly announced. here we are doing a deep dive into these human signals and hope this type of initiative is very complementary and can work together with the precision
and these are just two of many different initiatives that are ongoing. but i think there is a real opportunity for all of us to work together to tease out these because we are going to be integrating data processing huge amounts of data and then
exploring data in new and different ways. and we are very fortunate that nowadays we have new machine learning algorithms constantly evolving. the hope is to tease out what we should be doing because we think it will be make a difference.
and as i think about it, and as i move towards the conclusion, i like to think about what tools do we have to understand the population and precision medicine? and i know a lot of times if you focus on precision medicine, people frequently think it is
pitted against population medicine. but i think the two of them are intertwined. so if we think about all the new expanded data capture sources and many of these capture the working groups that you have, and is this on the left-hand
side of this slide, and then you think about all the expanded data sources and this is where pmi fits in. and in fact, one could argue that we are going to start to be able to study the entire world within the mobile health this is the a real interest in
international work. in fact many people predict that is where the biggest impact will be. targeting lower income where access different. so there is a real global opportunity here. but as we take this data not
only doe we need to think about known and unknown signals but i think we have to go back to the fundamentals of what it means to be epidemiologists and looking at nonrandomized exposures trying to get signals to see if there is a correlation. i hope we push further and look
for causation and embed randomized trials because then we will be able to truly know whether there is a link. so just because someone wears a smart watch and their health is better, is it that or is there something truly causative about having this closed feedback
loop? i hope that we can just describe for these very basic research principles. and i think if we do this right, then we will optimize population management and get to the heart of what you all are doing here today and really expand
precision medicine. so thank you for your attention and look forward to a really great day and a discussion that will continue. thank you very much. >> good morning. my name is josh deny and i'm a working group member.
i'm here to introduce section 1. thank you very much jessica for that great opening kickoff talk. if i could have the members of the session go ahead and come forward. i believe you all have talks. the first talk will be by dr.rhoda au, a neurology
professor at boston university and a senior investigator and director of neuropsychology at framingham heart study. she will give her discussions. you want to speak from here? okay. and i'm not going to go through long introductions on each of
the candidates. or each of our speakers today. they all have extensive experience with mobile technologies applied to clinical research studies. and that will give us more time to interact with them through the discussion.
>> thank you very much for inviting me to share with you our experience at the framingham heart study. and i think as many of you know, framingham started out as a study of heart disease but has over now nearly 68-year history has expanded to a study of
really chronic disease. so we really like to think of it as a study of health moving this is noted moving forward. >> can someone from the back advance? >> i'm still not getting anything. so what i want to start with is
tell you what framingham is doing in terms of integration with technology. and to start with, sort of where we are in the wearables is we are doing an in-home sleep study where we mail the dices to our participants and they wear it for two days and then mail it
back. here are the two instruments. we have them wearing an ecg monitor as well as an oximeter. and just to give you an idea of the kind of data we are collecting. we are collecting traditional what you would expect from a
sleep study in terms of breathing and then also because of the new technology, it is allowing us to look at novel analytics related to quality of sleep. i'm a neuropsychologist so i'm very interested in the assessment of cognition and
really knowing about how technology can advance this. this is not necessarily the stage of wearables but i think it's in the first step in moving us into a digital era. so in 2011, we started using the digital pen. at framingham.
this is a clock drawing test. could someone please play this video? and so, what we have done is we basically have substituted regular pen with a digital pen and i'm waiting for you to play there we go. you're playing it towards the
end but in any case, this is a window into the brain as the person is conducting the test. and this really gives us a life level of precision we haven't had before. but, i wanted to move beyond sort of tests in which participants use a written
response and we have now, in collaboration with my colleagues at ching wau university in peking union medical college in beijing, moved towards creating a platform for all our neuropsychological tests. so if you could play this video as well, please.
and in this case, we are taking sort of incremental step where from the scanned point of the participant it tons look the way they have always done the test but we are now digitizing it. and so, if we could play that. and i think that the understand goal here is to eventually get
to the point where we able to do cognitive assessment mostly, but this is the first step. i guess the video isn't working? all right. one of the things i like to remind you about framingham is that our value at framingham is in the data we have already
collected. as i mentioned, we have nearly 68-year history and so one of the things in thinking about how to we integrate technology into our study is, how do we also at the same time link it to what we already have? so we have already had a little
bit of discussion about the importance of big data analytics and we have a partnership now with thompson reuters. sort of one of our first steps is to start harmonizing data across different cohort studies and we are looking at framingham as well as the eric study
looking to how to harmonize our data, fuse it with structured date and what is called unstructured data using the thompson reuters proprietary tools and identify where these new relationships that are traditional analytics currently don't allow.
in addition, with thompson righters and with biogen, we have formed a partnership to look at multiple pathways to treat alzheimer's disease. i recently put out a paper where i suggested one of the issues with alzheimer's disease is we keep thinking about it as one
disease. when it is really multiple and we really look to big data analytics in order for us to determine what are those different treatment potential pathways for alzheimer's disease? now one of the things i like to
remind people, particularly when i'm working with the technology community, is the greatest consumers of health care are really the elderly population. and today, there is really an overwhelming amount of technology that is out there. and the other thing that i also
like to mention, and if we could play this video, is that the lowest consumers of new technologies are actually elderly. so is this going to play? we can build the devices. the question is, whether they will use them.
in this video that isn't playing quite correctly, it is really someone who has been given an ipad and is using it in a different way than it was intended. so where is framingham today in terms of really moving forward towards an e-platform for
research? and what we did was a survey of all of our surviving members from our generation 2 and generation three cohorts. and of that, 78% actually responded to this survey. now our participants are very active and they like to give us
a lot of feedback so they also took it upon themselves to give us some comments and to let us than as a whole, they liked the idea that we are moving into a new era, but we also get comments that suggest some are a little bit concerned about this new direction.
now among our respondents, we try different methods in terms of getting feedback from them. we used red cap but we used it as an online version rather than mobile version. we also need to follow-up with some in terms of phone and mail. among those who did respond, it
turns thought 91% actually have a cell phone, which was good news. but, 67% had a smartphone and not surprisingly, the ones who have the smartphones tended to be our younger participants. i think the average age was around 55 as opposed to the ones
who didn't have a smartphone and that average age was closer to 70s. but among those who did have a smartphone, many of them had an iphone. so, with these results in mind, my colleagues on the heart side decided to launch a pilot study
and they started out with 200 participants when the inclusion criteria was they had to have an e-mail address and i smartphone. and this is a three-month observational study. they are given the option of five different devices. and our initial results suggests
that not surprisingly, when we tried different recruitment methods, bringing people into the clinic, showing them how to use the device, led to greater usage of the device. both in terms of selecting more devices to use and then the data we have gotten.
but what i think is of importance is that 50% of our participants who are enrolled in the pilot to date are over the age of 65. and that is very promising. but i think that one of the things i always worry about at framingham is what we are doing
and more about what we are not doing. and as we are thinking about these e-health initiatives, one of my concerns is, the fact that we can't have people wearing multiple devices. so that is one issue. and then the other one is,
because i deal with-- i'm on the cognitive aging side so i deal with more of the elderly population, i also worry a lot about the user interface. and then framingham sort of, as i mentioned, is tied to our past. so i'm looking also to what
kinds of relationships need to be formed to move things forward faster than i might be able to do than within framingham? so i have a partnership now with shimmer. shimmer is an irish-based company. what i like about their
particular wearable is that is it customizable. it can be customized for a number of different metrics and the other important thing they think has been mentioned, is the need it be open source because we need to probably establish new gold standards as we move
into digital technologies. and then in addition, i'm concerned about that user interface. i worry a lot about participants, particularly older participants, having to interact directly, input things into a device.
i worry about the iwatch. my elderly can't even see it. so if you could play this video. this is an avatar that interacts with the participant it's scalable because it is electronic. it has been piloted in boston and it is currently being
piloted in china. this gives you an idea. one of the things we are hoping to do right now in collaboration with some hong kong colleagues, they have a research technology center there, we are really now looking to integrate. we want to integrate mobile with
smart home devices, feed it into this, what i would call this better user interface for my elderly so they can really use it in the home. and then in addition, we actually are launching a new longitudinal cohort study in china likely beginning of next
year and in this case, we are really integrating new technologies part of our new data collection process right upfront. and this is really a study of chronic disease so it's independent. in we collect the right data
upfront, it's independent of what your health interest is. and then finally, i have been very concerned about the issues of measuring diet. right now we have a number of instruments that are out there. i think that the goal here is that we really need to find a
way to look at nutritional biomarkers using a portable device similar to how we do with glucose. it turns out i learned in the past year that if you tell the technology company that this is what i want, sooner or later, someone will produce it for you.
and so i now am in discussion with two companies that have created nutritional device that is measure nutritional biomarkers and we are looking to start piloting those within the coming months. although i come from framingham and boston university, i think
what is very important as i moved into this arena is that we can't take a siloed approach. sequentially we built a network of collaborators both within the us and outside, particularly with china, to help us move this and i thank you for your attention.
>> thank you very much for that discussion. and next up is greg armstrong who is an associate member at st. jude's children's research hospital and also university of tennessee at memphis. and she going to talk about some of his work with children's
survival and the children's cancer survival study. my name is greg armstrong and i'm from st. jude children's hospital in memphis. and i'm the principal investigator of the childhood the curing of childhood cancer is one of the wonders of modern
in the 1960s less than 20% survive now more than 83% survive. we created a new population that didn't exist 50 years ago. that is who we study and follow and intervene on. looking forward by 2020, there will be one-half million
survivors of childhood cancer in our population all of whom made increased risk for certainly late affects driven by exposure to chemotherapy and radiation during their childhood. so the childhood cancer survivor study was funded as a retrospective cohort for
children diagnosed between 1970 and 19 ninety nine from 31 contributing centers, major children's hospitals in the united states and to be eligible, they had to be 5-year survivals. so 5 and alive. all major childhood cancers with
a detailed information on their treatment. so dose-specific, organ-specific for radiation and dose-specific chemotherapy exposure and then a wide range of outcomes as we followed them across their lifetime. over 35,000 eligible for this
24,000 participated and currently over 20,000 active participants. i show you that map to show you that our population is spread across the united states. this is not a population will ever come in for a one-time health evaluation at a clinic in
a uniform way. this is a population that we communicate with through some very old mechanisms. i call them our original three dimensions. these are patient reported outcomes. so patients participants have
the options of completing a paper survey sent by u.s. mail f they don't, we have a large telephone team who calls and follows up and they can complete a survey by phone. and since 2007, we added a web-based completion. so moved into the modern
generation and in the last year or two we asked some hard questions. what is the potential to now engage using mobile and fap let-based communication and the interesting question, what about direct assessment? this is what i blind our
committee what i call the fourth dimension of communication for our study. moving beyonds our original three dimensions. and in one slide i want to paint the vision they put forward for the executive committee. it's push, pull, and monitor and
intervene. push information to our participants, pull information back and then ultimately, monitor and intervene. so think of what we could do with cancer survivors as far as pushing information. we could become a resource for
survivors provide them latest news from our study as well as around the world. health information related types of updates. importantly we contain treatment many were treated at a very young age. paps tray have no idea what they
were exposed to. we have that treatment information and can provide them back to them as a resource. we can provide reminders to complete surveys or invite participants to additional studies through mobile-based communication.
but then pulling back information is incredibly important for us as a study. we can get updates on key health currently we collect second cancer information every three years on a paper survey. that means a survivor could have a breast cancer and die before
we get around to the next paper survey and that shouldn't be. with mobile-based communication we can get a realtime update on a new malignancy and validate that in realtime. we can allow them to complete surveys on their phone while transporting by train or rail or
bus to work. rather than having to sit down with a paper survey and we felt like we could do well through base communication. i draw a red line here because i think all these things we should have been doing a year or two we are rapidly catching up now
by adding at-base communication in our cohort. but it is what is below the red line that is critically important and the that's the ability to monitor and intervene on a remote cohort that previously we only have been engaging in by paper.
through mobile technology we can connect a sub cohort and monitor their health outcomes directly. so instead of asking them how is your activity? what is your activity? by paper. we can directly measure their activity or directly measure
their blood pressure instead of asking them. have you been told by a doctor you have hypertension? we have a current study. the ask study funded by an ro1, advancing survivor's knowledge of skin cancer. in a randomized trial we provide
in the third arm this study, a dermoscopic lense for their cell survivors are at greater risk to melanoma and perhaps by giving them the opportunity to connect to a dermatologist directly through their mobile phone, will increase rate of screening and sale exam and physician-based
exam. so the opportunity to monitor and intervene. ultimately one day i'd like to see we have a completely connected survivor cohort. so how are we going to do this and how have we done this? very simply.
for our at-base communication, we decided to connect and work with one of the largest at-base vendors sales force and we wanted to do this because primarily we needed to operate within a hippa compliant atmosphere and security health information for our survivors
and sales force has been doing that in a number of venues so we quickly built an app in a hippa-compliant space. but when it comes to monitoring and intervening, we cannot catch-up as a study, and understanding where technology is.
so we have been very quick to say we can't do that unless we join forces with someone who is already doing it well and that's the healthy heart study. here is the app we are in the final phases of development now and will deploy later in the summer or the fall.
it is the long term follow-up study app. it's just a historical nomenclature. our survivors know us as the long term follow-up study. you can see from the entry portal, you have to have a specific user name and password
we provide our participants. so this is their space only for their secure health information. we provide latest news on the homepage and then a menu board which you can see gives them a resource hub to key resource that is survivors need. a space for their treatment
where they can actually see the treatment that they were exposed to and push a button and e-mail that treatment information to their personal physician. and so on and so forth. so this is our at base it's below the red line that i want to spend the last few
minutes talking about. i think this is what is important for our survivors going forward. now we currently ask how active are you? what if we could do a 6 minute walk on every survivor no matter where they are across the
country? in collaboration, we are moving forward with co-enrollment of our survivors to have full opportunity for engagement of mobile applications such as a 6 minute walk platform or a photo diary, diet platform. instead of doing a food
frequency questionnaire, which is laborious thing to do on paper, now through a more modern dieted capture or photo diary, we can directly capture the food and label the food along with calorie content. so these are the types of opportunities.
so here is how we are developing our collaboration with the healthy heart study. their mission is to support multi-component research projects such as ours and to maximize our ability to use m health technology. what we are putting forward is
co-recruitment of our ccss participants into the healthy this is going to be provide us additional data on participant interest. will our survivors be interested at all? and more importantly, if fruitful, develop a sub cohort
for subsequent monitoring based and interventional studies. we started a pilot in february of 2015, five months ago and targeted 500 survivors for co-recruitment into the healthy here is what is important. the ability to collaborate across two major studies.
healthy-- healthy heart allowed us to taylor their materials and create a web portal specifically for our survivor participants. they are sheltered from most of the healthy heart messaging so we don't want our participants offing-- messaging from two cohorts and bombarding them and
fatiguing them. we have been able to shelter them. and then additionally we collaborated to share data between our two studies. so here are our pilot results to date. we targeted 500 patients see if
they will be willing to engage n5 months of time, 56% of this population cohen rolled into the this is phenomenal. i would have estimated it would take a year's time to get into the 60-70% participation window. in five months we are at 56%. not just that, of those who
consented, 80% have gone on to complete the first e-visit. so the first series of healthy heart questionnaire. they are engaged and moving forward and notably on the far right side of the screen, the number of refusals who said, do not ask me about this again, is
only 6%. so the summary of our proposal for our study moving forward is we are going to collaborate with the healthy heart study and add them in our upcoming competitive renewal as a new core facility. we have four core facilities handling radiation dosimetry,
biostatistics, and now a fifth core facility within our study to understand and lead a mobile technology. we will then move forward in recruitment of a 12,000 person survivor cohort. cohort cohen rolled into healthy heart and which i will call a
connected cohort. ultimately provide a large pool first and foremost for in-health based intervention studies. we can use this technology to intervene in this population to change the course of health in this survivor population. and finally in the distant
future i think it is realistic to have a completely connected cohort providing direct measures in realtime. so i think all of this is consistent with the nci's vision they laid out for two years ago for 20 21st century technology driven epidemiology.
in this meeting that happened two years ago, the call out was to include intervention studies to monitor across the lifetime to use novel technologies, be able to process large datasets and to be cost effective like everything in here in red that i think we are doing consistently.
so finally in closing, here and one slide where our key concerns is, i think this is the discussion points for future number1 is security. we have to protect the health information of our participants and do this in a hippa compliant space consistent with the
consent process. secondly, will it be used? this is a population used to completing surveys by paper and by telephone. can we induce culture change and is it sustainable? flexibility. is the platform we develop today
going to be flexible for next year and the next 5 years and 10 years? it has to be. expertise. quick to say we don't have it but we can engage those who do and collaborate. cost.
validity of outcome measures. my most important, i'm not interested in the newest latest greatest gadget. i'm interested in something that can measure an outcome in a valid way price and accurate. i want to know that if we are measuring blood pressure, that
we're actually measuring blood pressure not some other number. that's the accuracy. i need to also know that over time, in repeated measures, that those measures are precise and consistent. that is key. otherwise, none of this is worth
validity is the key. and finally, i think we'll talk about this much today is the ability to handle the data. we dent have that ability about you again through the expertise we are engaging, we should be able to. so thank you for the opportunity
to speak. it's a fantastic opportunity to be here with you. have a good day. >> thank you, greg. our final speaker in the session is dr.jorge who is assistant professor at the harvard school of public health and also in the
school of medicine. he'll present on nurse's health study 3. >> thank you for the opportunity to present about our work in nurse's health study 3. so, you may have heard about nurse's health study and nurse's health study 2 but not
necessarily about our study 3. what i'm going to tell you about is the most recent offspring of these chain of studies i right now it is an ongoing open prospective cohort study of nurses across the united states and canada. and we have right now
approximately 40,000 participants enrolled completely online. so, compared to where my colleagues said before, we decided to forego paper and decided to forego telephone and our initial engagement is moving forward starting from web.
now what i want to tell you today is how do we get here and what are our plans to moving forward with these cohort completely based cohort? of course, we had some experience in our group in identifying women or people in general.
and tracking them using questionnaires and paper. and we also had some experience in biologic phenotyping and obtaining biological specimens and obtaining blood and biological specimens increasing the position of phenotype on that end and what we wanted to
grow was, the electronic phenotyping in these cohorts. what are the type of information that we can gather from electronic devices from wearables and from your electronic fingerprint that can aid in advancing health research.
so, i'm going to focus today on three things. first, how do we get people into a study? what was our experience into getting people into a new study? how do we get people to follow-up over time? so if there is one thing that
you cannot buy it is time. so what has been our experience in following up with time? third, if we are able to get them to respond here on web or through phones, what is the infrastructure and what has been the decisions we have made in order to make the best possible
use of these data or at least what have been our judgments in trying to answer those things? so i'm going to start with the first two. what has been our experience in getting people into studies and making sure that we can keep up with them?
so, when we decided to start on the cohort or try to start a new cohort, we decided to pit against each other two approaches. one was going opposite to our tradition which was focusing on the health professionals and rather than that, we decided to
go for an hmo membership-based population-based cohort, because we thought this could have the opportunity for a scalability. there are existing hmo networks that have been used for research and we saw the opportunity to enforce scalability. these could definitely overcome
one of the major problems that we have phased before, which is the concerns about underrepresentation of minorities generalizability of finding from a cohort of professionals and health professionals in particular. and it also offered an
opportunity to do something that can be done very easily in europe and in the united states, which is to passively follow people with high quality data. so we knew that if we could get access to emr and insurance claim data we could potentially follow passively people and on
the other hand we decided to say let's try again with the health professionals. we have been successful with them in the past. let's see if we can be successful with health professionals again. what did we do?
the initial study which was the hmo base, we decided to do a pilot to test two things. one can we get people to enjoy the study? can we follow-up people for at least 6s? we haven't had any contact with these people before.
so, we decided to randomize them into two strategies for retention. one was to have them contact multiple times. the other was to offer them money to participate. and what happened with that. so the results in a nutshell, we
had a low enrollment rate of 5000 people randomized or 5000 people randomized in one of the four arms i he marginal response rate of 6.4% of all people impacted. now the good news is, of people that we were able to contact a lot of people were most of them
were eager to give us access to medical records and insurance claim data. so 72% of people who decided to enroll meaning they completed baseline questionnaire, were consented to give us access to their electronic health data. the bad news is that of those
who enrolled, only 63% had a 6 month follow-up. we were only able to keep contact with 200 people out of 5000 within six months. so, at the same time, we were-- sorry, only one thing we defined useful in these study was what happened with the intervention.
so, offering financial incentives decreased participation rate. so if you offered people money to participate, they were less likely to participate. and if you contacted people more often, they were more likely to participate.
so highest preservation rate were those contacted multiple time and not offer money. and we also looked at what would complete a follow-up and the only significant predictor of allowing us access to insurance claim data trying to gett-- which we interpreted if you want
to participate in a health study, you will participate in a health study. so at the same time, we were also evaluating the potential are going with health so we tested three strategies. one is repeating what we have done in the past just send blank
letters. the other one was send multiple contacts and the third was to let cohort build itself organically. so we identified 1000 nurses health site to participants that were actively engaged. we gave description of the
enrollment criteria and told them if you know somebody who might be interested in participating, let them know and this is how they can this is accumulative response rates over the first week of study. so the bottom two lines are the
blank mailing. so if you just mail people and ask them to participate, they won't. so response rates are 2% for just blank participation. then the person-to-person strategy was slightly more successful and that's what we
decided to go with. what we have been focusing on is outsourcing these and replicating these peer to peer recruitment by partnering with nurse organizations and by asking participants from nurses 1 and 2 and 3 and strongly in social mediad and we had a corn
about potential issues with under representation of minorities so we did a targeted mailings to potential minority participants and we essentially sent them this card so identified people we thought were african-american and sent them this postcard.
we identified people who we thought were either asian or hispanic and sent them this postcard. had information on how to join the study. and this is how our enrollment worked. so the peaks have coincided with
mostly mailings, mostly second or third party initiated personal invitations. so either an organization or group that somebody already has a relationship with, asks somebody to participate. the things that haven't worked has been advertisements so we
tried-oop facebook, google ads that hasn't worked so well for us. now, how about retention? so this is more or less diagram of participation in the first year and study worked for us. if you notice there is say lot of feedback loops telling us
people to try again and try again. until we get people to respond. and i think this has been the main key thing for us to be where we are. when we first started, we had a much lower intensity of going back to participants and that
had us about the same 60% response rate to the initial questionnaire that i showed for the other study. when right now by increasing the frequency about 75% responds to the first questionnaire and once people respond to the first questionnaire, they are likely
to say response rates go 70-80% after the first questionnaire. now lastly, how we have gone about the mobile part. so, the-- we decided to work into a three tiered-- not showing. so, just imagine there is text so, the bottom line is tier 1,
which said it was a mobile application. the second box was supposed to say tier 2 which is a commentary assessment data and for working on these two tiers we decided to go with product called spade system, an open source product for app development for health
information which was developed through the nih's small business program. and the reason we decided to go with these is essentially scalability and the fact that it can be modified and it can be-- we can change what we want to know over time and that has been
very, very important for us. and also battery life. it has a very, very low impact on the battery life. the third box that you're not seeing there is wearable we have decided to go-- devises-- we decided to go with fit bit charges.
the vane two fold. one is the granularity of data we are able to get from fit bit through the api partner. and the fact that the fit bit device as o bodes to other devices has addressed major issues that we had and that was mentioned before, validation.
so we want to know that what we are measuring is actually a trade that we want to measure and not just one random number. so this is data moves within our study and lastly, to finish major concerns towards implementation of the this first money but more than money
is the fact that we are competing for the skill sets that are necessary to make this thing a reality like our host second is the rate of technology change and trying to balance the rate of technology change against the long term partnership.
so for us, it is very important that if we start to measure something today, we want to know that we can still measure 10 years from now. and but that is not necessarily the same interest of technology. third one that had been very important and that was a major
issue that we decided to go to space is data ownership system. we could have gone faster on using that if we had decided to use other services. but a key issue is that many developers-- some services, has in the contract that the data belongs to the company and that
is a no-go for us. and we need to understand how stuff works. so, and that has to do with ideally we want to have as much raw data as possible although we understand that is not necessarily something that some companies may be able to share.
we want measures that are validated or validata, and as few black boxes as possible. and lastly, two things, building knowledge and comfort about these type of technologies has been a major issue for us. so it is to reflect, in my opinion, just lack of comfort
with this time of-- tape of technology and deploying large numbers of people. so they may be okay with a study that will do this with 50 or 100 people but if you tell them you're going to do with 100,000 people they get excited and start to ask a lot more
questions than you would normally get. and last data storage and data issues. security privacy that was mentioned and how do you manage these volume of data? so often to measure and retreat specific data if we are going to
get it from a large number of people to make sure we are store it somewhere. that has also been a major issue. and also we think that going forward, there is going to be a major need for standardizing processing for these type of
data and that we currently do not have. so, that's it. thanks. >> thank you very much. this is the first session of the day. so we will now go through a session where we'll ask
questions and just moderate and talk among ourselves and then towards the end of this we'll open it up to the audience for maybe the last 15 or 20 minutes sore so. so thank you. i detect a number of themes in this discussion.
one of the things that maybe interesting to start with is enrolling patients and so i think each one of you talked about mode else of consenting that were electronic and i guess have a couple of questions in that vain which would be how did you manage identity?
how did you do the consent? verify it is who they say they are in terms of sharing health data that could protected by hippa? especially when you talk about connecting to hr for nurse's health study? >> so most of our participants
right now are web only not necessarily participants in have provided spec which require additional level of identification or those who require m health data which requires an additional level of identification. so the way we have dealt with
that is asking our irb to continue the traditional model we have for paper based studies in which they agreed that if somebody returned a question that was implied evidence that they were interested in participating in a study so just translated that to the visual
world and said if we send them a questionnaire and they enroll, we get data back from them and that means they are interested in participating in a study. and the il. rb was comfortable with that. as far as validating who they are, we have done that in two
ways. one, is tried to obtain social security numbers or other data that can be linked to national index for other social security base outcomes data. now we decided purposely not to do that at the beginning of the because we didn't want to
freak-out people saying tell us about you and by the way what is your social security number? and your mother's maiden name? so we collected this type of information to make sure that first we are collecting among participants who are most faithful participants and second
to decreased concerns about privacy information. we think that once somebody has been in the study every two or three years, provide information 6 months or more often, they will feel comfortable providing us with social security numbers. one thing that we have not been
successful at and we keep trying or that i want to become successful, is in getting a target medical record and insurance claim data from hospitals and from insurance companies. given we are not starting from an hmo-based study.
i think the main reason we have not been successful is that many hospitals have not transitioned or from the model of paper-based request for information to an electronic base for request for all of our transactions are so we can get electronic signatures and authorization
from our participants but when we transmit that information to a hospital or to a health care provider, they will say no. we really do not know that these electronic signature corresponds to this person so we are not going to release information for these people.
>> so we for a unique situation in that we have a cohort that we have been engaged with for 20 years and have a are rapport and relationship with. as we move forward with our at base communication-- app based the app is going to provide treatment information.
this is pro protected health we mail out a letter with a personal identifier code, access code, that goes to that participant. they use that code to access the app and then they have to then enter a second piece of information that only we know
such as their birth date. so by entering those pieces of information, we link them to their portal until a unique way. this is for us, not having to develop that expertise was important. so we by engaging with healthy heart, use their e-consent
process to consent to the >> so, our relationship with our participants is even longer. right now, our original cohort has largely passed away. this is under 50 that are alive today but for children, grandchildren, that are enrolled, gen 2 and gen 3, this
has been their whole life and their family has been part of this study. so we don't have the same issues in terms of recruitment because we are closed cohort that is generational. and so, we have what i would say is a very compliant participant
base. the other thing is, because they have been in the study for so long, they signed multiple consent forms so that is not an issue and when we went into the realm of genomics, genetics, we had to go back and re-consent people with the idea they need
to consented for things we can't yet anticipate. so, we have that grandfathered in. we have proxy information. everybody had to identify proxy. i do all the cognitive dementia side so we have access once they lose the cognitive period of
time tow provide consent. so, i think the other thing we have that is really important that has been said here is we do really have a personal relationship with our they expect us to know a lot about them. i happened to go into their
homes and they expect i know everything about them. i know everything about their family and everything that happened to them. so we may be have a different kind of relationship and i think that that speaks to why with our recruitment base when we used
our in-person contact we were that more effective. >> i agree with all the points. the hope you're hearing from everyone is the new technologies will make the consenting process truly informed. so i know many of this in the room have consented patients
20-page forms with clinical trial. while it has to be at a certain reading level, the reality that people are truly investing this information seems be likely and corking with consenting wizards and user experience groups which has been a lot of fun
personally, really trying to think about the end user and how that information is being generated, i hope we push the boundaries on irb. mostly to make it user-friendly for our participants. >> i think each of you talked about different mobile
technologies that could be used and most cases, these mobile technologies, it seems like they were standardized. either across a custom app that was going to be uniform across do any of you plan to use just a variety of different types of technologies in trying to
harmonize them on the back end such as fit bit and some other way of assessing the number of steps per day? >> so, i agree that that is one of my major concerns right now is that we have multiple devices sort of moving forward and we need to integrate better.
so i sort of take maybe a two-step approach to that which is one, is as we move on the back end, is using the big data analytics to help us harmonize data into a more centralized form about the other piece is we need to think about integrating our devices themselves.
like i mentioned, we cannot ask people wear multiple devices and part of my thinking about what devices to use are part of the reason i selected shimmer, is because it's an open platform and so on the one hand, it's customizable to pick up a number of different metrics which one
i mean from the standpoint of participant, they just put o it is programmed to measure different metrics at different points of the day and all the participant has to do is wear it, dock it to recharge it and wear it again. so i think that is vealy
it's open platform and can integrate across other devices and i also mentioned i have collaborations going on with the research institution in hong kong around technology development. we are now trying to integrate so in terms of mobile devices,
smart home devices, and then also user interface that i think is scalable and then also can speak in my case, a lot of interest in the elderly population. >> i'm not particularly in letting devices drive what we connect.
i want the research questions to drive what we connect. in my mind we want to ask what are the key-- hypertension. for survivors of childhood cancer chest radio therapy and hypertension compounds their risk for heart failure. so when we need to measure
hypertension. so we won't do that across the 10 or 12,000 person cohort but we want to do it in a focused way, in a way to intervene. so selecting a sub cohort of that population for a randomized intervention trial where hypertension is the outcome.
so i don't see deploying one device or multiple devices over the entire population but more creating a platform for intervention-based type systems. >> so, i think we are halfway between those two points. so, there are a number ever traits that we are interested in
measuring we know some of them can be measured with wearable some of them could be measured with smartphones or with just built-in sensors that most smartphones have today. the main-- the main drivers of our decisions to go with space for the application with fit
bit, for the wearable device, is not because we don't think others are not good, especially on the wearable device. -- so it comes down to the issue of we don't want to measure something for the sake of measuring it. we want to know that when we are
measuring something, we are measuring what we think we are measuring. so we decided that at this time, at the state of technology or at least at the state of technology that is very viable in the scientific literature, we could only be certain that you could
measure sleep and physical activity with fit bit it's possible that other products can do the same or better. it's possible that you can do better with other devices for other trades but we don't have any evidence. we don't have any evidence they
will work so we cannot publish papers. just imagine writing in the method section. we use this device and then the hocus-pocus in the background happened and then we got data. i don't feel comfortable doing that and in our group we decided
that before going to these specific devices or apps, we need to know what we are >> and i think the biggest challenge is to try to tease out won of the items i started to talk about. the known versus the unknown. it makes the best sense to use
the best device-- so for example, we know atrial fibrillation is a problem. but chances are there are a lot of signals that are unknown and probably even better detection so, as a cardiologist, we look for heart failure. i could list off what i think
are going be toot five best things a mobile device should measure but that is based on my experience. and it may very well be actually the number of people i see or how much noise i'm exposed to or all sorts of other variables and i think to try to push the
envelope, we need to consider some of these unknown signals, especially in the context of research settings. again, it's a much harder thing to do but i think that is something we want to keep in mind. >> so, just to add to that.
what i want to remind everybody is framingham started out as a study of heart disease and we have now grown into a study of chronic disease. i always tell people i'm very fortunate t turns out what is bad foryour heart is bad for your brain.
our dementia studies didn't start until 1976. the diagnostic criteria for alzheimer's wasn't published until 1984. i agree we have to know about what i always talk about saturday balance between novel versus innovative.
so novel is sort of more based on precedent where we have some bases for understanding and moving forward. but i think we have to think about the innovative and the innovative, i always tell people, for something to be innovative, you can't have heard
it before. therefore there is really no base for us to make that so we have, if we look at framingham as an example, it really is about, if you measure the right things at the beginning, even though you don't know your measuring them, they
could be really important. >> can i make another comment? so, one of the other reasons we decided to limit the number of things that-- the devices we are measuring now is that we think we need to gather americans precisely identifying these unknown signals with a
standardized device that we know everybody has it. but, obviously the idea is that the experience we gain in more controlled setting when we have a single device that works, can be used to build a platform where you can integrate data across multiple devices and
across multiple types of electronic data moving forward. >> i think you could go thinking about the different devices and validating the measurements to the perspective of thinking about blood pressure cuffs. so you mentioned hypertension and blood pressure.
we have a wide variety of blood pressure cuffs that can be used within the hospital and we believe they work. assuming that they are meet certain standards, standards and they get validated and our patients measure their own blood pressure and i believe their
blood pressure, they tell me they used a certain kind of cuff and then i check in my office. so maybe we are early on the face of this but you can imagine a variety of device that is are validated being acceptable. and if you think about google, your large data across many
different sources often seems to win over a smaller dataset with a lot of precision. and health care i think we are usually more comfortable with a lot of precision in huge measurements than maybe more in precision over lots of measurements.
and i don't want to spend too much time on the topic but any brief thoughts on that question? >> so i think you highlighted three key themes, one is the openness around the data, the second is collaborating to make sure we understand the signals and they are identified and then
further validated and then we take it to the next step to make sure they are potentially causative. so i think that is important. and then the final one, just goes back to our original theme. we know what we know. but we don't know what we don't
know yet. >> i want to remind us in terms of what we think we know. we know it based on old ways of gathering it. so, when we think about the gold standards that we are trying to validate, we have to remember that we are validating them
against the old metrics. and so, that is where i really worry about us tieing ourselves too much to what we already know. i like to use blood pressure as an example. we know blood pressure fluctuates throughout the day.
we don't know actually how much it fluctuates. and certainly not in a normal most of the people we monitor very closely actually already have high blood pressure. and the reality is that we don't go from particularly in the area of chronic disease, we don't go
from being normal one day to being not normal the next day. so it's something that happens over time. because we haven't had the technologies up until this time to measure something even as simple that we think we know a lot about in terms of blood
pressure, there may be now with these new technologies, a new arena of what we really think is going to be gold standard. so that is probably one thing i'd keep in mind. >> one of the things i was glad to hear from everyone was the importance of thinking about
validation of the meaningfulness of the outcome that we can measure something highly precisely and not necessarily have clinical impact. and of course i always jump to thinking about the cast trial. and suppression of other things that we thought were meaningful
and the ekg obviously was not helpful in that study that caused access death. and so, i think we'll find lots of things and everyone has echoed this and i don't know we need to spend more time on it. were there any particular unintended benefits or
unintended harm so far seen in your study using mobile >> so, one of the unintended benefits that i hasn't really anticipated turns out when you sort of start to merge technology with your research interests is that you create new ip.
so since i sit-in intel right now, that might be of great is that when you create this new ip, you create new opportunities for products and services. in my initiative with china, both in terms of the mainland and hong kong, they actually look towards the development of
national cohort study, not-- they are interested from the health perspective certainly but they are also interested from the economic perspective. they think that this is going to help drive their new economy. so my own personal is that suddenly i have to give
disclosures a way i never have before because i'm now part of initiatives that are moving into the commercial market. so that is one really unexpected. i think it's a good thing. >> i'm not sure we are far enough along to answer that
question well. obviously one of the downsides potentially could be breach of information in a way we never had the opportunity to breach information before so i worry about that. that keeps me up at night. but i think there is tremendous
benefits that-- our population, i mentioned this the pilot using the dermascopic lense, sex ro1 our participants are jumping off the shelves to get this device. so there is say novelty and interest factor in a i have high quick participation with that study that amazes me beyond any
other trial we have done within i worry about the staying power. we are in the milling of that first year. will they be consistent with this novel device over time? we are seeing slowly in the first year some wanning of that. so i do worry about that but i
think the verdict is still out on that. >> so, we have various concerns regarding data security. i think that that is definitely the biggest thrash we have to deal on a daily basis. and another unexpected problem that we had is providing
feedback to participants on a timely manner. so timely means something very different if you are dealing with somebody who is communicating on a smartphone versus somebody who is communicating on a letter. and we dealt with that very
early in our study when we asked screeners for depression and anxiety. and we always leave space for our participants to add additional information they want to oftentimes participants leave information about providing more details than we wanted to know.
but, we have had at least one instance in which one participant gave us or wrote in the open comment section that she was having suicidal ideation. now, that is one way to deal with the timeliness, perceived depending on resources and how
you deal. so if you deal on a paper-based questionnaire, which were we developed our experience, and you're checking paper questionnaires months or years after somebody has checked these, your responsibility as an investigator is very different
from when you're gathering information and people, even if you tell them that you will not be checking information in realtime, people may have the expectation that you will be checking information in realtime. so that had us change some
procedures in the way in which we delivered the questionnaire united states because we were not prepared to address these situations. -- questionnaires. not with our infrastructure. it is exceedingly costly to manage for these type of
interactions. and another side benefit that we have seen is that people are much or more willing to share information than we thought they wanted to share information. so, they wanted to tell us a lot more than we thought they wanted to tell us.
so, one characteristic of our study is that we embed within our questionnaires, a lot of branching to allow personalization of questionnaires and a lot of personalized routes within the so, essentially everybody in the study has their own timeline
depending on their experience. most of our participants, so our average participant is professional woman of 30-35 years of age who has never been pregnant. so, we are now entering the peak years of these women and one of the characteristics, one of the
things we allow branching for is for pregnancy and for people who think they want to get pregnant or actively trying to get those are best participants. people who actually-- the people from whom we get the most information about. we also have additional
branching to assess occupational exposures and to assess detailed family history and one concern that we had, at the beginning, is that we were asking for too much information and the feedback we get for people who actually do go into these questionnaires, is we are asking
them too little. so that has been quite possible. now whether we want to keep asking more and more detail we haven't decided. >> you raise an important point that we discovered as well. long before this becomes cost effective and you can imagine
with mobile health-based communication, we might need less staff. long before that happens, it is cost intense. because we are doing all of our data-- day-to-day activity and on top of that our new twists new questions and realtime
engagement it's become cost intense and it's unbudgeted. so it is an intense process. >> i would say that too. i mean in our recruitment, the in-person contact seems to be the most affect and i have that's not a funded initiative on our side.
but i think it is surprising how much of our participants, because many are older, are using technology at some level which i think is very, very encouraging. and then their willingness. they are very patient with us too.
and so, as we throw new things at them, as long as we take the time to explain them and walk them through it, they seem willing. i got to tell you, one big thing is we are drowning in data. and we do not have within our current funding mechanisms the
capacity to find the expertise we need on the analytics side to make use of it. so more than anything, we can collect all this, but i have to tell you, we need a lot of help in terms of analysis. i mean, framingham has that problem without adding the
additional technology into it. >> i would highlight that. we think a lot about mobile technologies and today the focus is on that discussion. but as i was thinking a lot about it, the synergies between the technology go way beyond just the devices or the
molecular toolkits but it's the back end to analytics. and partnering with strong software companies and groups that have thought a lot about data and also have access to very large data centers. it makes sense to extend these discussions in those kind of
technological spaces as well. >> let me open it to questions from the working group and to the audience. >> so in addition to the terrific work you're doing with mobile technologies, you must have the made a lot of floss engaging participants in
thinking about the questions you study and the ways you study them and the ways you give data back and those are all really important to us. so could you talk about the role of your participants in what we however all call governance? >> so i'll answer that first.
so one way in which our relationship with participants has changed a lot has to do with the expected speed of feedback. so, when you move from a paper-base or telephone-base under act that happens occasionally with a web beige or mobile base that can happen any
day any time and when people may be in different stages of their study at any one particular time, giving feedback and giving good quality feedback becomes increasingly difficult. so, in that sense, participants have become or have changed the way we operate.
we can no longer operate with the same infrastructure we had for paper-based questionnaire because that doesn't meet these needs. the other thing that participants have done is that as over time, participants have actually shaped how we ask a
question. and they have shaped some type of-- our decision to ask or not ask, for different types of so, i already gave the example of the mental health issues. so, we became aware that we couldn't deal with realtime interventions nor we were
interested nor i think nor it was responsible for us to pretend to be web psychiatrists and to try to deal with these type of issues in realtime. so, it changed the way in which we approached how we assess mental health issues. but, so that happened in the way
of restricting things on shrinking its way we were doing it. but it has also changed the way in which we ask things. so, in terms of adding more so one thing that i think-- here is an example for that. it has to do in our particular
population, has to do with occupational exposure. so we started trying to ask occupational exposures of these women in part because we thought that it would be something of interest for participants and would keep them more engaged. but, since we keep opened fields
to gather information on how they think we are doing, as far as asking questions and as far as asking whether or not the questions that they want to get asked, we have expanded the assessment of occupational exposures. we have expanded the timing of
when we ask occupational exposures so we only used to ask pregnant, now we ask non-pregnant as well as women trying to conceive. and it has led to refining of some questions. so we in moving from the paper to the web, we tried to maintain
as much as problem. as we moved to web, we relate some question that says work well in paper don't work well on the web. not only because the format allows you to gather additional information that you didn't have on paper or better information,
but also because you're dealing with a different generation that each if you want to-- even if you want to have data across multiple cohorts, that might not be the question that you're asking for a younger cohort. >> is there any other viewpoints on governance you want to
discuss? >> participant engagement is we have a limited number of survivors f i put out a fit bit and the instructions aren't clear, they may leave the study all together. and that is mission critical for us to be engaging in a way that
is clear communication. so, for our app or any other survey, we have a patient engagement process. we'll pilot that in a small group of participants and get feedback and we 38 help to guide us as we go forward. so engagement, we cannot be in
the mind-set of those 300 million people out there. if someone doesn't want to participate, we will find someone else. we need everyone. >> do your participants, are they involved in either selecting or prioritizing the
topics that you will study? to say what is going to be in the next round of grant proposals we write? do your participants engage in that process? >> no. yes and no. langley no because we have been
collecting the same information consistently now over 20 years so there will be a base set of information that we will continue to collect. and that our additional studies above and beyond that are targeted at research questions. health related research
some of which is driven by the feedback but most of which is not. >> i would say they are not actively involved in deciding the science but they have the obviously the option of opting in or out. we have many studies.
our participants are being used to contacted about different studies that go beyond the heart everything i do is not related to cardiovascular disease. so, i think but we also have multiple engagement with our participants and they do give us feedback.
with the sleep monitoring study frons, that was a new way for us to do sleep study. they had previously participated in the sleep health heart study which was very labor-intensive and required that we come into their home. so, the feedback we got was just
that they are very happy that we are not having to come to their home to do it. but two, they sort of give us feedback even in terms of yes, you didn't give us enough we had to change the brochure a few times. they were very-- our
participants were very engaged at the time when we moved into genetics work. at one point, we had brought in a private company to do potentially the genetic sequencing and our participants objected to that and so we pulled out of that.
so i would say that where we deep them informed of what we are thinking about. we take their feedback and incorporate it into how we are going to move forward and it has at least in one instance, short of changed the way in which we did move forward.
>> great. in the interest of time let's go through questions and answers quickly. we talked a lot about i was wondering if you folks could also talk about providers. the importance of engaging providers.
you're talking emrs and potentially actionable items. >> so we have an advantage at framingham because you can't go through medical school without learning about the framingham heart study so that makes it easy to engage in providers. one of the things we ask our
participants to do is sign a release to get their medical records which we have been doing for many, many years now and we contact the providers directly. i think it does help that we are in the media as well. a lot of our studies are published.
our participants are proud to be in our study because their physicians will sayi these are all things. so we have a little bit of an advantage on that front. >> just quick comment on that. i think there is the providers in a study and then there are
providers more broadly as we start to have all this data and putting a physician hat on, physicians don't always want more data right now until we understand it. and also making sure this is something where physicians and providers and health care
providers are part of these decisions such that whatever comes out is really-- i call it a health care provider support tool it's not something that you hear a lot of people around here talking about how there is no need for physicians and we hope that it just helps make
everyone's job better but i think engaging now is critical. and then one comment to the engaging participants. obviously we are now talking about crowd sourcing of clinical trials. we have to be aware that the paradigm is changing.
patients are organizing themselves which is great. and what does it mean to be a data donor, which takes it a little bit outside of the hands of what we traditionally think so we are going to see some big changes in study design in the years to come.
>> i want to add my voices to thanking you are if i series of informative presentations. you know of course we are struggling with thinking go how to assemble this cohort of a million plus people. and we want to be able to support great science.
and part of that has to do with a general consensus that in order to do great science, we need to ensure that it includes the diverse aspects of our population because we know that disease disorder and related conditions and risks are not randomly distributed in our
we know they systematically vary by age, gender, sec, race, ethnicity. assuming we can accomplish that at the cohort level, in the data acquisition process, we have similar concerns about the same extent to which these factors may be at work in terms of the
differential participation of the members of this cohort. so, in your answers to several of the questions, it's been from the perspective of the participants you already have in yow cohorts as o modes to how you engage them. dr.chavarro's report is very
striking low rates of participation, raise great concerns. because we know even the conventional methods we have today in terms of recruitment and study participation, we see enormous differences by those very same factors.
so, my question of you is, what kind of guidance can you provide us based upon your experience to date if you were thinking doing your study in order to ensure the kind of diversity and representation to do great science with the applications that you had?
>> so i actually confronted this very question in thinking about how do you develop a china is 1.3 billion people. there is no way to get representation from anyone one study group. so our strategy has been that we are defining a core protocol.
we are saying that we know sleep, physical activity, nutrition and environmental exposures are very important to health. all health outcomes regardless of what your interest is. and we are really mobilizing around that.
so we have a cohort study. that's what you do. you have to establish the baseline and then from that i think, then make if widely available. this is all about data sharing, right? so if you put it in the hands of
many, then i think then you have the individual investigators start to move out and find those represented populations. >> how do you take it and generalize it us? >> so in the u.s., i think it is the same concept, right? we started with a cohort study,
framingham. and i think that was sort of my idea is sort of how do i build. framingham is not representative today like it was before. and so, i think the idea here is, it's the same consent. if you establish the same buys protocol, and widely share it,
we talk about team science and data sharing, this is really where we have to go. we have to put it in the hands of individual investigators because you're going to need that sort of-- certainly at the beginning, that personal touch to really drive it forward.
>> just quickly, we are fortunate in our study we have good distributors in participants and nonparticipants. so we can describe nonparticipation. that is are have very important to describe the bias that may be
potentially in your study. >> so, this is an issue we have paid a lot of attention to. especially after our experience in the hmo base enroll am. so, as far as trying to improve diversity, what we learned very early on is that you need to bring expertise for pem who know
how to do specific things. so, we know how to start studies but we don't necessarily know how to make advertising for joint studies. so we have been very fortunate of being able to work very, very closely with dana farber health communication score that has
done a fantastic job in signing our communication tools and redesigning our messaging to participants and also helping us figure out how is it that-- what is it we can do specifically to reach out under represented minorities. if you look out at how much does
it cost us to recruit caucasian female as opposed to minority woman, it is incredibly more expensive to recruit her, in part because we have put a lot more effort into doing so. so when i described the targeted enrollment, so what we did was essentially we played junk mail
and racial profiling. what we did was we obtained licensure rosters from every single state in the country. some states will sell you race so here are the names and if you want the race information we can sell that to you for another however many dollars.
so, we purchased that. and for the states in which we knew the race of the potential participants, we target them specifically. most states will not sell you. so what we did was used a combination of data on zip code and-- where these people were
residing and demographic data on the racial composition within that particular zip code to try to guess the race of different people and then we did these targeted mailings. we might have been wrong but as far as trying to get a particular minority where these
people were, that's why we didn't include any specific messaging in terms of words. it was in terms of images to try to garner. q. for participation. >> i think we are going to go for about four more minutes. so i'm going to resort to having
done before and ask for lightning round-style questions and answers. i know you have been standing there for a while. >> i'm a private investor and a member of the angel capital association particularly the life sciences syndicate within
that group. i have just one quick comment and then a question. the comment is, on security it seems that security has been mentioned by everyone as being critical and i don't think it is just critical for the scientific data point of view.
but also from the patient's confidence point of view. they know that if they give a fingerprint or type in their social security number, it's not very secure. now there is a company i should mention, i'm not invested in it, called, i verify which is 19.99%
rely built they takes or uses the camera and a smartphone to look at the blood vessels in the eyeball and distinguish between identical begins it maps or changes over time. it cannot only-- so you don't have to do anything. just look at your phone and it
verifies you and it can make sure that you are-- it's continually you who is interacting with the phone throughout the interaction. it might be something to think but my question is this, you all mentioned some degree of pushback as you have begun to
adopt mobile technologies. could you just quickly summarize what those key sources of pushback are? >> i'd say the number1 factor is sage. what the is age of your participant you're targeting? >> and what they are objecting
to is my question. >> in our case, we have a very young population. so they are very comfortable using mobile devices. so in our case, this comfort >> i agree. question over here? [ off mic ]
thank you. john white from omc. it's an actual question and i'll make it a rhetorical one since our time is limited. in your opinion, across the precision medicine initiative, what is the right level of collaboration on shared data
infrastructure and analytic tools and things like that, to enable the goals of the precision medicine initiative? so that is-- you don't have to answer. just think about it. >> one word, two word answers. >> team science, multiple pis
on every grant. by the way, and then data sharing for anything that doesn't or isn't specific on your aims. >> collaboration is the word. >> yes and i would add collaboration between industry and academia and it is going to
be key to move this thing >> collaboration sounds good. and i think for the last question here since you're a member of the working group. >> i was trying to be a gentleman v a lady to the right standing there first. so i'll yield to her and i'll be
quick after she speaks. >> just a quick comment. in the workshop number 2, we discussed the use of omics and the exposome and the environment and many other things that can be captured by group analytical tools. i want to remind the panel that
perhaps a cross talk between all of these tools that can bring measure, the metabolome perterbations all across sleeve cycle are things that can be measured by measuring chemicals. the diet, and the composition of the diet to get the quantitative measure.
we have analytical tools and this is being vetted by the metabolomic community. connecting the dots is going to be important to derive knowledge not only data. and connecting this to the monitors will be great. >> and i would take it a step
further and say, now is the time to start thinking about how to integrate all of that data because i think we heard from the panel there is the collection and then there is the appropriate integration. >> you were a great gentleman. i'll let you have the last
>> thank you. roderick pettigrew from nih. i had two questions. they are both simple but one is longer to answer so i won't ask that one. just the first one. greg mentioned validity of data as a key concern.
and i think that this is a question that often gets a free pass among the public. there is an assumption of validity there. and those of us involved know case. can each of you talk about this issue and how you have addressed
it or addressing it in your cohort studies? >> i'll start. and again, i plead that right off the bat, we are not experts so we had to collaborate with those who are. so with the health healthy heart study, we are counting on them
as a core for our group to-- we are not going to put forward a device that doesn't have adequate validations. for us it is leaning in on those with the expertise to validate before we move forward. >> so my strategy is do both with the same time and collect
the old data and new data and compare. >> same thing here. so, the way we have done is start with a device we know has been validated but that doesn't mean we need to stop there. so, one concern that i have heard from industry is that the
level of information that we would want for complete validation saying give us raw data, gets in the way of ip issues for our company. so, what we may need to find is a middle ground where our academic investigators feel or have the ability to work with as
much familiarity of data possible without making public trade secrets of companies that have a very strong monitorization. thank you very much for a very obviously an energetic and entertaining session. >> and this is gwen jenkins.
we will be taking 15 minutes break. so we'll reconvene at 10:30. we are off to a great start and we are headed into a high-energy session on mobile technologies for participant engagement. i'm bray patrick-lake, one of
your co-chairs to the nih director on precision medicine and i'm delighted to introduce to you all today, dr.maribeth gandy. she goes by marry bath gandy. she is @maribeth gandy but not @maribeth collman. the director for the interactive
media technology and for the insitute of people and technology at the georgia institute of technology. go dogs. just kidding. and then we also have dr.bonnie spring and she is the director for the center for behavior and
health and professor of preventive medicine and psychology and psychiatry and public health at northwestern universities. we also have dr.dire a chief data science officer and former president & chief operating officer at weldoc.
and then we also have cathy saigona from the healthy heart study and she is a study participant and alliance steering committee member. with that, i will turn the podium over to dr.gandy. >> hello, everyone. so, i'm a computer science and i
study all the cool things you can do where wearable technologies, reality and i work at the human computer interaction level figuring out how to apply these technologies to application that is are effective and people want to use.
and so, a guiding principle of work that people like me do is user center design. and at the core, this is about not trying to force the users to conform to your system or your but rather truly understanding them and the context that they live within to design as is that
is appropriate for them. and so, a couple of things i want to highlight. one is when we are talking about mobile systems, and especially wearable systems, these are supposed to integrate into your life as you go about your business whether you're at work,
playing with your kids, socializing with friends, exercising, and so, those systems you have to be willing to wear them, you have to be willing to interact with them. it has to feel comfortable on your body. it has to look a way that you
find appropriate for your style, for the community you live in. and wearable applications are meant to support you in your primary task. it's not a typical computing application where you're focusing your whole attention on it's supposed become part of
your regular life. this is a big challenge then for making a system that elegantly integrates into your experience. so, another point i had on that previous slide was about all the different stakeholders in these kind of systems. so yes there is a user.
the person you're giving this app to or asking to where this but then there is the support network around that person, their friends, family, the child that is doing it support for them, the doctor that is receiving the data, and the community at large.
so, if i'm using some system like this out in my life, my coworkers, my friends, random people with me on the bus are also going to possibly experience part of this it says. especially in health systems where people may not wish to disclose the details of their
health status. oh, you might want to design a system that isn't discrete but presents itself in a way that is like positive, and firming other person that is using it. now so, this attempt to design this individualized solution, you first have to understand the
ecosystem that this person exists inside of. so, what are they doing in their daily life? what kind of work do they do? what kind of food do they eat? what kind of entertainment do they par take? what issue the cultural norms
within their community? and again with this health application, obviously there lots of privacy and security concerns some of that is meliorated by the amount of value that the system provides. but it's particularly tricky because there is a lot of
aspects of our health and wellness that we may not want to advertise to our friends and colleagues so there is a major leak. there is-- we heard about all these great technologies that are out there for sensing, so collecting this data.
we collect it and then we have to figure out how to present it either to the caregiver, the medical professional, the user. we presented the data but then, especially the user, there is a process of sense making and then developing strategies based on their new understanding of what
this data was telling them. and the real challenge then is making that leap to positive behavior change and long term adherence. we all seen various studies showing the drop-off of use in a lot of these wearable technologies, for example.
you get it for christmas, you use it for a week and you lose it, you lose interest. and a lot of that has to do with, if you're asking someone to really heavily engage with this mobile app or wear this thing every day, it has to be providing them positive value
dalea. its fact that is helps to manage diabetes may not be just enough value. so an example is under this hood, your standard health monitoring system so you got an older adult, you're monitoring through smart home technologies
their activity level, their various health statistics and you can imagine presenting that kind of dry prescriptive way. instead that data drove this very compelling positive application that populated this picture frame that would be at the family member's home.
so if gave a vague sense of who the older adult had been doing over the past few days. and the point wasn't to look at that and diagnosis a problem. the point was to trigger communication between the family. so, trigger a phone call,
trigger a visit. or some other sort of interaction. that was really provided a lot of value to this older adult. instead of feeling like it a medical app watching them, it's like, this is a really great specious where i feel more
connected to my grandkids and people are calling me more. so without the saul it will disappear, it will break, they won't understand it, there isn't motivation to continue with it. so you need provide that value. and this came up in the last panel.
is there a one-size-fits-all all solution? and unfortunately, i don't think there is. you can have this great device or app but if you're talking about drastically different user groups, that one-size-fits-all may not be appropriate.
so my pictures here, they are all drastically different people wearing google glass and that baby is my baby so i'm not violating anything. so here is an interesting piece of wearable technology in drastically different situations from a police officer to a baby.
so, there is all these aspects we have to consider beyond-- there is the basics that technologists like me think does this collect valid results? where does it stream the data to? what is the update rate of data and that kind of thing.
but we have got to consider especially with the wearable and mobile experiences. what is the context of this particular person is going to be using it in? and if it's something that will be visible, what kind of statement is it making about
them to the outside world? what are the social conventions? even if it's just a mobile app. it doesn't seem all that fancy. is this something i'm instructing pull out constantly while i'm at work? is that considered appropriate in the environment that i'm in?
a lot of these wearable things, the problem is, depending on like the community you're in and the type of work you do, you might not feel it socially appropriate to wear an accessory like that. so, you really have to have different sorts of solutions for
different sorts of people. remember you're using this system as you go about your daily life. what are the cognitive is sensory requirements of using this thing? so i only have a limited amount of attention and cognitive
resources and visual field of view and spice my body and when you're introducing a system, you're taking up some of that. and you can go so far that you're seeping-- keeping them from being able to go about their regular life or you're making them uncomfortable, like
physical, socially, so you have to consider all these things. so to conclude, i won't go through all of these. these are interesting lessons learned from related research in this area. for example, others looked at asthma management in kids.
ass adult, we might think, here, we have a great system. use this inhaler and you have this mobile app, this is going to be great. except that the kids are being made fun of on the playground for how it looked when they used that inhaler.
another example, becky ginter has been looking at nutrition education for people in lower socioeconomic levels. these are people that live in food deserts. if you give them an app that teaches them to drink kale smoothies, it's not going to
have any affect because the people in that community cannot access kale. they couldn't afford the kale. it's not like socially maybe the norm to eat the kale. so, this order of application that she built teaching you how to make good choices with the
food that is available in your community. this is all in this theme of nudging. so there is a lot of research now showing that if we want to affect positive behavior change sometimes the best approaches is like life touch.
so helping-- there can be a lot of really complicated algorithms behind-the-scenes but knowing when to give this just in time information and help someone just nudger them into making a better choice. and that can have bigger affects than the application that swoops
in and tells you this negative feeling like you're doing this wrong and you're unhealing healthy and need to run a marathon and drink your kale smoothie. back to my point about not a one size fits all solution. so sms use for asthma
low-tech solution. you get a text message t asks you for like a bit of information about your condition that day and really the best affect from this is it makes the person themselves reflect on the state of their health that day. when you take that same system
and apply it to people with congenital heart disease, it does not work. because it made the users feel upset about being reminded of the condition they have. so, you can't just take this one technology or this one application design and apply it
uniformly. so, to conclude, what we know from various research studies of wearable systems is that not only does a participatory design approach work in terms of creating better finished system and we heard that already today. not only does he it do that, but
the people who are the participants that feel like they were active partners in the research and the design, have a more positive view of the system at the end and continue to be more engaged with the system later. so, one way of getting people
more engaged is by having them feel like they are a real contributor to the research. so with that i will conclude. >> now we'll hear from dr.51y spring on social support apps if are engagement. for engagement. >> thank you thank you for all
the hard and very worthwhile work you're doing on this and thank you for also asking for input. so, what features should a mobile application have to ensure long term use? and much of this you know. i'm sure you heard again and
it needs it needs to be-- i think there is one thing that is critical and underlies everything sells that's the value proposition. it's got give someone something that they want and need. given the fact this is an observation study, one of the
things you can give people is best if it is in realtime. you can probably ask them to do more if you remember the bushed en proposition. ask people to put more into something if you give them higher value and reward back. and i think that is the sweet
spot that is really important to remember. just taking an example, i work in the prevention space. i work on health behavior change. one of the things i work on is weight loss so here you see what we used to do to have people
accomplish weight loss, we have them track diet and activity on paper and pencil records and now we give them apps which still we can't-- they still have to enter their diet but it can track automatically their physical activity and stand on a scale that wirelessly uploads
their weight. so does this make any difference? it does in good directions. so if you look at self monitoring adherence, this is did people track their diet, exercise and weight every day over six month period?
you can see that with paper and pencil, we get a third of the days tracked. if we just give them the technology which gives them feedback in realtime, we go up to about half of the time for diet and physical activity. we actually go over 90% of the
days tracked over 6 months for weight. and this is just the feedback. so the feedback is something good. and it's something you can use would adding social support improve clinical outcomes? so we have been looking that the
a bit and this is an app from a study that we just completed. we had people in a weight loss intervention and we had them competing in groups of 8. and we wanted to know any giving them information about their group members would help them succeed in this self tracking.
we asked people if they were willing to share their weight loss with people. nobody was willing to have their weight floating around in cyberspace. but, what they were willing to have was their self monitoring so we developed an app where in
traffic light colors, they are seeing whether somebody has been tracking their diet in take and wearing their accelerometer and so it's pretty cool. people said they liked it. did it work? well, it seemed to work for about half of the people.
for that half, they behaved like we thought they would. they said i was really jazzed to know what my peers were doing. i wanted to reach out and support somebody if they were having trouble. i wanted the support myself. but then other people said i
really didn't identify with those people and this was sort of burdensome and the other thing it did, it created a problem that we had some people participating in the app and chatroom and nobody was responding. so it was actually isolated that
person and making them feel very bad. so our thought was, okay the problem is these groups are too small. if we have just one person speaking in a chatroom of 8, that is too small. so we went bigger and got data
from an online weight loss network, 27,000 people. what we looked at was, did the social support in this digital online space work better? the first thing we found is only 11% of people used the social security function. only 11% of them made any
friends at owl. if they did, it made a huge difference. in this site about 40% of people went to the site once and never came back. half of them were gone by two weeks. if somebody made one friend, 90%
of them were around six months so, the friending, the socialization, is clearly related to adherence but we don't know the direction of the affect. >>> here is the relationship between social network and weight loss and what can you see
is, if they made any friends, they lost more weight. if they were connected to the giant component of the network they lost more and if they were connected to the connector hubs, those who welcomed people into the to cheer them on, they did really well and there is a dose
response relationship between how many they were connected to and what their weight loss was. but, the problem is, we only have about ternal% of people that benefit. so how could we improve upon that? -- about 10%.
-- i think what i finally realized is, i have been thinking about this like a researcher who thinks of myself at the center of the universe. i have been recruiting people and trying to stratify my sampling and control the randomization.
i think what i need to do is to go to the social networks in the communities where people already are and let them bring their own network. so, i think this is related to a question i heard raised in the last engagement session which was, how can a precision
medicine cohort engage resilient people and how can it address prevention? and a number of us are thinking that establishing a college subset of that cohort could really go a long way here. but enrolling college cohort really requires to you address
the value proposition. i'm launched on a trail we are about to begin this funded by the american heart association that addresses a little known but important health problem. and that is that during the college years, college students lose between 13-20% of their
long term cardiovascular health. the reason is that they pick up these chronic disease risk behaviors like they gain weight. they often pick up smoking in conjunction with their drinking. they become physically enact and i have their diet goes to pieces and by the end of this, they
have lost a lot of their cardiovascular health and these habits tend to persist. so, we are about to launch a trial in which we are intervening to try to prevent this loss of health in half of them and the other half are addressing other risk behaviors.
so, i thought this should be pretty straightforward. i'm a health interventionist. so i'll just go out on this. i thought i better spend a while getting to know the college so we spent the last year doing that and learning what nay and what i that telling you they
tell us they value is most of all, academic success and future professional success. saving the world and maybe have a little bit of fun. their college administrators in turn also have things they care mostly they care about academic success because they want
successful alums and also care about the health problems that wind up getting them in the newspaper. some of you may have seen the newyork times article today on suicide in the college binge drinking, sexual violence. i'm sorry to tell you, that if i
walk-in and suffer my treatment, there is nobody who is saying, where were you when i needed help with controlling my fruits and vegetable intake? just like i'm sorry to tell the commit they is there is probably nobody in spite of the positive feedback who is sitting around
saying, i just can't wait until i can enroll in this cohort. people have their own agendas. they are going about their lives. and so, it behooves us to figure out how to align with what they are trying to do. this is what has gotten us very
heavily into an engagement solution. what we are doing is citizen science. we are truly codeveloping our intervention with the students. we have no choice. we would be fools to try to go in and do anything else.
who are having them because they don't understand why they should care about these health behaviors, we are having them do experiments on themselves to look at the impact of their physical activity, their diet, on their academic performance, and o. their pattern.
something else they care about. and on their stress. we have computer science courses engaged in developing apps for they said look, we can't exercise because we have no time. so they developed an app that tells them, when they have free
time to exercise and suggest what they might do. a number of us, we have a workshop coming up this thursday with a computer scientist and health specialists working with us to try to establish a college student cohort, multi-site, to look at this problem.
and so my recommendations to you would be, keep the apps use age, simple, personalized, and introduce nosty from time to remember the value proposition. it's fundamental. the burden has to be really low unless the reward is high. the driver has got to be what
the participants are trying to achieve. it's important to capitalize on the existing social networks to recruit. and it is really key to collaborate and coinvent with the community because they have to drive it.
>> now we will hear dr.armand iyer. >> hi. guys. it's okay. so, i'm a wireless guy and i have been in wireless for 20 years and then something funny happened when i developed type
ii diabetes myself and i joke what the two things that define me, wireless and more sugar in my blood. what i like to do is share with you the journey we have taken in the outcomes we have achieved and share with you along with the way frameworks we
established that will help this entire dialogue about what we do at precision medicine and the so the honest truth, as a patient on a day-to-day basis, i have to manage everything, my glucose, my medicine, my diet, exercise, stress, sleep, smoking, and is there a thing
called life that gets in the way every now and then. at the same time, i consider myself a little bit of a nerd. i'm a data scientist. so i take my data into my doctor and a graph it. but what is the doctor going to do in a two minute office visit?
the human eye isn't trained to look at a series of numbers and say oh, my god, it's time to adds 10 more units to this patient. and so, at the same time, doctors don't want to be bombarded with data. if you ask an overworked and
underpaid doctor to do one more thing no matter how sexy it is they won't do it. you have this crash and there is a lot of stuff happening. a lot of buzz going on around. i have new meters and they countdown from 5 instead of 6 and i'm like, i have a hole in
my finger. it doesn't matter. so they are innovating in ways that don't matter to me. we asked the question, are we solving the wrong problem? i love this picture because the arm is broke but this is intact. are we focusing on the wrong
problem? the problem is actually two fold. the problem is first that the right data is not getting to the right person at the right place at the right time. but that say problem that every other industry solved.
that's called connectivity. and you can do it with all the security and privacy and know cringes and nonrepudiation and all that good stuff. great. the harder problem is, how do you take that data and convert it to information, knowledge,
actions and tangible outcomes. and tangible outcomes for me, maybe very different than they are for bray or anybody else and for the different stakeholders in the health care ecosystem. one of the things that is required in this transformation, people call it behavior science
s context. if a tell a patient who is naive to insulin, and they are afraid of taking their insulin, maybe if i show them how to inject or show this them is what you do. maybe they don't want to take it because grand 345 died on insulin.
well she start today too late. you have a chance to live. if i show them that 6 months afterwards, it's annoying. i got it! things have to be contextural to them and their med history and medication regimen and psychosocial preferences.
so, what we did was build a that says had four parts to it. there is a patient coach and we are talking about realtime. and the patient enters their parameters on their smartphone, dumb phone, any phone t has to be i device they use. it could be a t.v., ipad, pc,
let them use the device is most convenient to them. but when they enter that parameter, they get real team coaching on what to do. my coaching may be different than sparrow's coaching because our history may be different. -- coaching so it is tailored
and personalized to them. we had patients they entered a good glucose value and you say way to go richard. they say if you give me another way to go message i'll throw the phone away. the next lady says, if i have a nice day, can i have a picture
of my grandchild show up on the phone? that's what keeps me motivated. so you provide realtime contextural coaching and then provide clinical decision support. we talked about it earlier this morning.
people say you know what? mobile health and digital health will take doctors away. when you're sick you don't want to see your phone, you want to see your doctor. but the doctor has to be a contributor and participant in this equation.
so what do they get they get a nice formatted analyzed report at their beck and call that suggests doc, here is where they are and here is what changed. shear what you need to do but you're the doctor. do what you think is right for your patient and i'll talk to
you about the outcomes and then the expert system that ties it together and there is a little growing piece if i would call, social. because at the end of the day, sometimes as you start to look at the ethnic groups, especially south asians, you look at the
hispanics, we listen to one another sometimes more than we listen to our doctors. but if i can find a recipe that work for me, that works within my parameters and my constraints, then that social networking and that concept of getting information from my
diabetes peers is positive. so when you take the solution and turn it its side, what do you have? four modules is cool because this stay platform of sorts. i can expand this to asthma, rheumatoid arthritis, oncology. modules stay the same.
content changes. rules change. but there is medication management come is required for us in diabetes and every other chronic condition. this is symptom management and lifestyle management and this is all aided by this internet of
things. the faster and easier i can get this data series of sensors, that's what we do. and then the last thing we do is measures we can take numbers that we can act upon. in our case, it's blood glucose and pressure and weight.
asthma it's barometry and ovarian cancer, where we can have a number to hit and a marker to hit, we can then provide tailored feedback to patients on what to do. so what i want to do is introduce these two frameworks. which is what are we doing in
the realm of data collection. i would assume all data in health care, the vast majority sits in this bottom quadrant. data coming they low frequency, say once every 6 months or 12 months if it's claims data, it's coming from a lab or system and that is necessary but it's not
sufficient if you would in mathematical make the outcome. what digital health does is expands 3 new vectors of data. the first load frequency data but it is coming from a patient. that is, their medication profile, it may change because these are somewhat progressive
deceases. it's their psychosocial profile. it may be their own preferences, the people in their social network who may come and go and at the same time, you get high frequency data and you first get high frequency data from the system which is cool because
even the absence of data, on day 3 if i don't use this system, it will say, you know, i notice you haven't nut data for the last-- so even the absence of data it is a prompt action. or you have repeated hypoglycemia. i'm glad you're checking but
let's work on why you're waking up every morning at 60. could it be you're taking too much long acting insulin the night before or did you skip a meal or things like that? and then lastly, you have the data coming from the patient on a day-to-day basis.
-i'll share data with you. it's myth busters. and of course, these data have to fulfill these five vs. they all have to represent a variety of all the parameters they said. the more retro genius data is the more value.
the more value you extract. there is a volume you have to consider because you need to make sense of what i'm going to show you with pattern recognition and there is the velocity and veracity to make sure it's correct and then that's how you extract value f
that's what we collect, what do we do it with it and i like to introduce this framework i introduced at the joint session with the nci and nih, called idea. it says i first inform. here is where i know my data and my outcome and this is math 101.
reporting. show me how many times a patient did this or males did this or asians between the age of this age on this med regimen did it's 101 mathematical manipulation. i inform. once i inform, i then discover.
this was dr.mega, who spoke earlier. the concept of the unknown. can i learn something from that data? can i learn something that i otherwise don't know about a cohort? for example, why is it that all
of these indian males who have a certain set of practices because they fast every fortnight, why are they on a-- that makes have you hypoglycemic incidence? that's the wrong drug! so their social behavior informs the doctor, this is the wrong therapy for them!
ut put them on something else, the and then once i discover something of interest, i come down and extrapolate. a build a predictive model. today i can predict after 10 days of use, on the 11th day to the nearest hour, if a patient is going to go
hypoglycemic and out of the 274 billion we spend in this country on diabetes, 47 billion is unnecessary hospitalizations due to hypoglycemia. i'll show you a chart. and then lastly, once i extrapolate, i then adapt. that's how i change and this is
the point that bonnie just made. refresh it. give them something new. update it. but update bites this journey of -- based. if you put this together for a patient, what does it mean? it's the data and all the data
they collect that gives them information and knowledge and makes them do certain things to keep them in the safe zone. is the absence of evidence the evidence of ab innocence if you not to be a player you burt have out ands proper research otherwise it's a gimmick.
so can you imagine for diabetes the gold standard is a1c and it should be less than 7 and every-- heart attacks, kidney fail, blindness, amputation, stroke. the fda harold a drop of a hat. .5. when we got a 2.reduction they
said what are they doing? swallowing the phone? no they are doing what their doctors asked them to do. and that is a huge finding. this is 2-3 times the value and when you compared to other drugs, you can see how dramatic the affect is.
at the same time, if i show you some-- i was listening this morning and i added this. i'm going to take 60 more seconds and then stop. which is, real world data. so this is a cohort in maryland. you can see the numbers 600 plus patients.
and look at the engagements. okay, they entered their meds. med entry is a huge problem. they are tracking meds. but look at the third one down. notes. you when they nut a high blood glucose after lunch, they are telling you why.
they are telling the system why because i'm glad you mentioned it's okay it was 300 but let's work on that and they write, i won't have that extra slice of pizza. i'm going to try harder next the free notes are averaging 83 words.
they are writing paragraphs. we never thought they would do so myth buster. guess who is our power users? all above 65! doctors say i'm not going to prescribe it. they are not going to use it. they are the ones using it most.
perhaps they feel their morbidity and maybe they don't work and it's the only way they keep in touch with their grandkids today. nobody does this anymore. another one, people thought this was goo good for people on the people on orlando's are the
ones left behind. the oral guys are using it just as much if not more and then look at this curve, that's the number of hypoglycemic and hyperglycemic trends in this by the way, everyone of those episodes costed between 14,000 and a thousand dollars for visit
or er admit or something like and look at the trajectory of this curve and think of what the cost savings in the-- direct cost savings of that is. so, yes, is this the future? i think so in many ways. this is for real. and then you know, i'll end with
this, which is i'm fascinated and i'm humbled to be part of this team and we as a company being part of this team, but let us not forget apart from the research and the president's initiative, having a chronic condition is lonely. and people go into this downward
spiral, this abyss of misery because they are alone and that has all kind of negative consequences and if technology can help abate that, if it be hymn-- can help give them a place to go to we have done something of value. >> so now we'll hear from cathy
who is our healthy heart study if for green. i'm very pleased to be here today to talk about a subject that is very for and dear to my heart and i don't mean that to be a pun. i'm going talk about the study itself, the healthy heart study,
how it has changed me as a person, and then talk about some of the lessons i learned. a little bit about me. i am retired principal. i worked with 7th and 8th graders for many, many years of my career and i have since retired and i am now an
adjunct professor in the school of education at a local university where i live. anyway, because i became so interested in what was going on with the healthy heart study, i was asked to be part of the healthy heart alliance. and with that, we have looked at
how to engage patients and how patients really could support a lot of what we do in research. they have a voice. and so i'm quite happy to do that i was asked to represent our group in february, late spring, and i learned a lot about what i can do as an
individual but what we can do as a healthy heart study. about 10 years ago, i was diagnosed with atrial fibrillation. i didn't know anybody who had heart disease. nobody in my family. it was me.
and after working with local cardiologist i was referred to ucsf, 100 miles away from my house. and so to go there, i will to have considerable information to take with me to make sure that that 15 minutes that i saw my doctor was going to be worth my
100 mile drive. i analyzed how kids learn and grow. i looked at barf, i looked at student achievement, i look at absent rates. and so i was really into the global perspective of studying i knew data was power.
and i wanted to make sure that i could bring that same power, that same rate back into my own life. and take it to my doctor. so, with that, i would go to see my doctor and we talked about starting or he talked about starting this particular study.
it's an online cohort study. you have to be 18 years or older and you have to have an e-mail address and you have to have a heart. and that's it. it can be a healthy heart. it can be one that needs a little help.
but those are the only requirements. the healthy heart study was conceived by three practicing doctors and researchers. one is here today. dr.marc pleacher. dr.jeff olgen and dr.marcus, the three that started this.
their overall goal was-- to improve cardiovascular health. that's their broad general consensus. how are they going to do it? collect big data. keep costs low. we all talked about in the last couple of hours, about
collecting data. for me, living 100 miles away, was considerable costs for me and for the doctors faves they wanted to collect my data on a regular basis. the cost of gas. the cost of my time. the cost of the nurse's time to
do my blood pressure. the cost of the lab in order to take my blood. so, all of that can really be dispensed when we look at what we can gather online. we talked about smartphones several times and i just had another statistic.
this particular one, greatly affected me that is one that morgan stanley did that 91% of all the phones that we have are within 3 feet of the individual, 24-hours a day. when i first started this, that was not me. i was in the nine%.
9%. my phone was in my purse and where was my purse? my file cabinet. where was i? among all the kids. so i had no belief. when something i started asking my friends, many of them work in
the private sector, many of them work if schools. the younger the person was in my little study, the more that was accurate. and then i started talking to folks who were working in other districts. yes, they carried them right
they kept track of things on their phones. so i went, i'll try that. so i did. i'm one of those 91%. what got me up today was the app on my phone that says wake up at 6:30. so i was pretty excited.
let's talk about the healthy and if you choose to join the study, it's the very first screen you're going to see. and i find that it is one of the screens that will open up an amazing world to you. it did for me. there is a little button, a
little video button off to the side that will bring you in. it will talk about the study. it's pretty jazzy. some people would say it's kind of sexy. i was enthralled the first time i saw it. so, let's say we are going to
join the study. so, let's do that. i'm is not going to talk about irbs and consent. i don't have that skill set. so the first thing you're going to hit will be a size of surveys and screens for you to select. this is just the flowchart.
you're going to be looking or giving them basic information. detailed demographics about you. your activity and well-being. habits and lifestyles, and some of these subsets under each one of these categories are surveys that are tried out in the practice of medicine.
your medical history, detailed family history. it was interesting when i had to start talking about my parents, my brothers, my sisters. how detailed. hi to call my brother. once that is all done and you finished your detailed family,
then for me, i had atrial so i had a couple of more surveys triggered based on that. look at all the blue. this is a pretty exciting map, i think. every country that is identified in blue has participants in our today we have 28,700
participants and currently, 21,841 consented participants. that's pretty exciting. i think healthy heart well on its way to beating heart this is basically our-- where we are and this is to support actually dr.icer, statement that most of the people are
between 60-69. that's me. and so it's pretty good to see that we are doing pretty well. and you can see between male and female, we are about the same. some of the smartphone apps that the healthy heart looks at will take care of behavior, activity,
communication patterns, locations, we have tiny survey that is pop up every now and then. hospital detection. 6 minute walk study. the diet. i believe bonnie talked about that and a social graph.
and activity, sleep, weight, blood pressure. etcetera. this is one of the screens in the healthy heart and i think it is probably one of the most important ones for me in that the study is agnostic in thought towards what device you're going
to use. the other thing that is pretty exciting is i can pick and choose. and i did pick and choose. i don't use every one of these. and i can choose my level of engagement with the healthy one of my friends doesn't have a
smartphone but she is very engaged in this particular so, she only inputs things through her computer. i input more but that's where i choose to do that. one of the things i do is a blood pressure measurement. as a heart patient i do have a
problem with high blood pressure. i must say this has helped me tremendously with my pcp and my cardiologist by the fact that i can keep track of my blood pressure on a regular basis, it's easy to excuse downloads into my phone and it is stored
somewhere in the cloud and i can take that information to either doctor. a couple of weeks ago i went to see my cardiologist and made that 100 mile trip. bart was out part of the day so it added to my stress on the free way getting there.
and so, by the time i got there, i'm 15 minutes late. my blood pressure a little bit high. so anyway, i get there and the doctor says, so how are you doing? i'm a little excited here. so, he read my blood pressure.
yes, you are. and so, first thing he did was say, can i see your log? absolutely. so i pulled up my i-health and showed him my log. but then more importantly, i said, i want to show you through my electronic health record, i
jumped into there and i showed him the last five times i had been to a doctor and had them take my blood pressure. in the past, that probably would have been well, cathy, i think you probably need to adjust your medication. at this point, we didn't have
to. because he could see that things were pretty okay from the information i was giving him. the second one is-- one of my favorites. since i have arrhythmia, i use the life core ekg it's very simple.
every now and then, i get excited about it. just put your fingers on the diodes there. 30 seconds later i have a print out. that's not me. i'm 100% pacer free. i'm very-- since i'm pacer
controlled, my heart will not go below 60. so that 59 is not mere. anyway, next one was the app. ginger io has several things working in their particular app and this is one. ucsf wants to know when you have been in the hospital.
so, this alert will come up and basically, it will ask you why you're there. are thru because you need to be worked on you there because a friend or family member is being worked on? i lived right behind the surgical center.
so, i got this app probably six times a week. and finally i called them and said, you know, i do live-- so it is basically a geofencing type of thing. so it pinpoints me and so it thinks i'm in the hospital a lot.
so anyway, we squared that up. but the thing that i liked about it, it gives me a little, hey you're doing good. so, to sum things up, i was happy to hear that some of my conclusions met with some of my colleagues here on the panel. i want an app that is easy to
read. i want an app that is intuitive. i don't want to have to spend a lot of time. some of these apps can be used as individuals. i prefer to work as an individual. i'm not a joiner.
and some of them you can join with. and then the other is the visibility factor of these apps. about 5 days-- every 5 days, i get an alert from the life core. if i have not already put in an ekg, it's going to ask me to do so.
so i'm more in inclined do it because somebody is watching. and i'm going to comply because basically i want to do this. this is my contribution to the relevance in my life. i'm more engaged. i'm a participant in my own i'm a better patient.
i come prepared with data. so it's not just me and my 15 minutes with my doc. but i come prepared with lots of i probably wouldn't have been where i am today without belonging to this particular and then the last is, a little bit altruistic.
there are lots of people who have been in studies such as the framingham study. lots of people have been out there to help pave the way so that i can be healthier and so, if i can contribute, i'm going to do it. and it's kind of fun.
>> okay. so thank you to all of our presenters. we are going to do a bit of rapid fire discussion. we have questions to go through with you folks and then we want to move into our audience i think we touched upon a lot of
these. so, keeping in mind that the working group has to write a letter when we leave her today. we want to pick your brains very concisely and rapidly. so, what are the most important features to incorporate in a mobile application for the
precision medicine cohort to i think each of you touched upon a few of them but just what is the most important. >> i guess i'll start. i want something that is easy to use and doesn't take much time. >> and then building on that, i want something that collects
things that are relevant to me and that will help me at that the problem i'm struggling with, and then also help me with if it's a recurrent issue, help me with solving that problem over >> and i want something that gives me feedback in reel time on something that i care about.
>> i think the fact those were 3 different answers shows there may be isn't one thing so it's about having something that, personalize itself based on what your values are and your psychosocial desires and such. >> all right. so how do we personalize
something for a million people? >> i think this goes to the issue of engagement and to the fact that you can't. we can't as researchers, because we don't know ow audience that we need the engagement of the people that we are trying to help who really need to take a
hand in personalizing things for themselves. and i think that this is what is going to lead to some of these new research paradigms where we start to engage people on collecting data on themselves and learning how to organize those data in way that is they
find valuable. >> so, this is an opportunity for all of us. if you put on your data modeling hat, for a second, we are enamored with segmentation. i say this is the segment i want to cater to and then i define as the reverse of that is called
clustering, says i'm not going to segment. i'm going to observe and see the natural diagrams, heat map of sorts. so i'm just going to collect i'm just going to collect. i'm not going in with any thought of this or this.
what we'll find is that there is several diagram intersections that occur like a heat map. and the heat map then tells you that is the balance between scalability and precision because everybody would say is it precision or personalized? precision has to work for either
individual or groups of individuals. but it can work for groups of individuals t doesn't infer in-- n equals 1. so i would say that because of the sample sizes we are dealing with because of the people who use these technologies f we can
collect this data, and if we can use clustering, and there is 1000, go read the book by duda and hart on how to cluster, versus about thinking about it this way. it's a paradigm shift. i think we-- the sky is the limit in terms of how far we can
push the envelope on this precision thing. we will also get scalability is. we won't have to build solutions for n number of people. i really was intrigued by a pint that you made. it verged on the precision public health.
because that's part of what we are trying to achieve here as well. you talked about south asian men and in 14 days you know. and basically you discussed how their social environment interacted with their medical and how we need to incorporate
so how do we taylor that? and i want practical suggestions. how do we taylor to that to particular communities. to communities in malaysia for example? >> so we have done it in a couple of ethnicities and suck
groups. one the asian communities and the idea here is that you use surveys and use periodic surveys in your application to collect preference data and monitor the change in that preference data. one of the techniques that is widely used is, you may have
heard of a company called insignia out best in oregon, judy hubbard is the ceo. and there is a patient activation measure, the pam score she created. and the pam score looks at these different attributes and in her case, she uses attributes to
determine a score of engagement. in our case, we use the attributes to determine what kind of configurability and what kind of content do i need to deliver to that person? because if i know, for example, that they are going to fast or if i know that they are just not
going to exercise, then i made the treatment pathway may be different. i'm not going to say, go lose relate. i know this person is never going to get on a treadmill. let me at least help them with meds, maybe they will swallow
more pills. then you go that way. it's a combination of having them tell you in their manner and building those profiles and watching them over time. >> i have a quick question for cathy. so when you're consented for
this study, did they present it as solely a research study or some implication that this would be a health management tool as well from all the great date you would be getting? >> i would probably say both because i worked closely with one of the doctors who divides
i took it for me because i was interested in improving my so i took an active participation in that. and if i'm going to give them data, i need use that data. >> so, what features or functions would you recommend to provide participants with more
of a sense or ownership over their data and over the project? >> began, because i'm so big on function and value, i really think that we need to give people the opportunity to perform experiments on i think that is really the only kind of research design that
matters to most people. they don't care about the average outcome. they care about what is going to happen to themselves and so, i think that we need to move towards building apps and systems that give people that capability.
>> so, 95% of me agrees with you for all the obvious reasons. at the same time, there is a piece of the generality that makes sense. its it goes back to that diagram concept. because if there is n number of people, you're not going to have
n number of nonintersecting circles. there will be some dimension of commonalty, whether their race or med profile or whatever it but this concept of do it for me. i'll share a real example. one of the ways we trained some
of our models was we had a grant from maryland and we got those continuous glucose monitors and we gave to patients. and on one hand they had a dense every 5 minutes blood glucose sample coming in and then we were training our sparse data system to interpolate and
predict the continuous glucose. i asked the guys if i could keep my sensor afterwards. it's fun. so first thing i did after the experiment was done, i opened up a bottle of red wine. and i poured myself my red wine because i love my wed wine.
and literature says-- for me, why, i don't know. it was stunted and i was like i owe you one. but then, there is a south indian dish i love and i made it and it went straight up. i'm like okay. we're even.
and then i think whole neat my body for whatever reason has a higher glycemic response than cornflakes. why again i don't know but for me-- so i think there is a certain amount of-- right. >> there is a sensible middle ground that actually relates to
how we think about evidence-based medicine. when you think about what people will want to do to help themselves, there are things that have a research evidence they work for most people. that doesn't mean they will work for you but it's probably a
reasonable place to start. if that doesn't work for you, then there is probably a second line something you want to try. but the important thing is, you try it out and you collect data on yourself. so that you over time, are tailoring what you're doing and
really, in terms of research and the new research designs, this is what we are doing with adaptive research designs. we are figuring out those decision roles. and so this is why i think that the ability to have these digital data creates the
opportunity for a test bowed move some of the research methods faster and forward in ways that we need. >> i wanted to-- if you come that the from a different angle. so we talk about wearable sometimes the solution for continued engagement and
adherence isn't to provide more value in the health space but to make that device useful programs in a totally different domain. so one of my colleagues at georgia tech, was developing a necks um surf for medication compliance, a holy grail sort of and it's one thing when you
apply that to like an older population, people are happy to wear it. but then he started looking at what about teenagers that have had organ transplants? so it's very important that these kids be taking their antirejection meds like super
critical they take them regularly and at the right time. the problem is, the teenagers are not going to wear this thing on their neck unless the other kids at the high school are wearing this thing on their neck. now this is a really tough-- i
don't have the solution for but the way that you would make this a success is you would make that a cool thing that maybe it is used for something different and we brainstormed about, could it be a game control or a social networking input device? the kids at the high school have
to think of this as like a cool thing that like connects you to cyberspace, not a thing that monitors your medication. >> the healthy heart alliance sponsored a conference called the summit, last november, and anybody who was part of the healthy heart was invited.
we had about 150 people who gathered over the course of a day and a half. and during that time, we were doctors, patients, like me, researchers, we had a multitude of people coming from all over the united states. and part of that work was to
come up with a study and in fact we came up with 11 studies. and we divided up into groups. these people have never worked together. we were all strangers walking into a room. we had some phenomenal people who were the facilitator who
made it all happen. but we got up, we voted with our feet, we moved, why interacted, and finally divided into self divided into 11 cohorts and from there, we wrote 11 different studies. one of those studies on atrial fibrillation actually was
continued on and flushed out and went for funding unfortunately it didn't get it yet but we are still hoping. >> so, maribeth said something that is really important. i want to amplify it, which is, blockbuster drug of the 21st century is what?
to engage the patient. if you get them to do what they need to do, 90% of our problems in chronic disease go away. and then the remaining 10%, the research will figure out cures and things like that. and the trick then is how do you get them to engage?
and pick something that is relevant and interesting and fun if it's a sport enthusiast, make sure your app has sports analogies or feedback if it's a somebody interested in horticulture it has something to do-- whatever. see what i'm saying.
that's a really important point you made. >> so if-- i'm struggling with the scalability and we have to deal with a million people and everybody is a individual. are there a few basic strategies you can deploy early on? like if you can't have a million
different personas, are there like certain things you can start with that are just kind of like, if you do 4 or 8 or-- how do you attack that problem until you see what the heat map reveals and then adapt that later? >> so can i give you the nerdy
answer first? so, for all of you who remember high school algebra, if i have a matrix of 100 people, and let's say i have 10 variables, pick your favorite 10. maybe it's gender, maybe it's disease they have, maybe it's their favorite sports team.
join know. could be anything. when i look at that matrix and i take the ranks of that matrix, it tells me a lesser number. so if i have n patiented, i may have n over 2. maybe n or 3, n our 4 groupings. there are other mathematical
modeling techniques to gate dimension down quite low. so even if you have a very large n, you can very quickly arrive even with just a handful of attributes at meaningful cluster that say that is where i'm going to start. i'm go going to assume those are
my clusters and then i'm going play the nearest neighbor game. you know, you're the n plus 1 patient but you look just like cluster 4. okay, you're in cluster 4. your the n plus 2 patient and your dissimilar to everybody. are you your own cluster?
and you start another one or maybe grow around you? i think there are ways to do it's just math. >> yes, i am my own cluster. thank you for asking. sherry? this is really fascinating. i think my question is a
follow-up on this. what happens to ethnic identity or other major categories of identity communities on the heat map? do people emerge into those? or does that suppress on the individual-- i'll put myself totally in another space?
>> i'll respond. so i'll take this one personally. i hate filling out forms. it's the ethnic asian. what is asian? i mean, somebody from tie juan and vietnam and india are very in every possible way you can
figure out. so i think you don't want to-- when you don't have to average, and that's the beauty of technology, you have the ability to collect things. averaging past filters and smooths out things and removes the variables that discriminate
and you want discrimination here because discrimination allows to you keep discrimination the mathematical since allows to you keep specificity around that so i think in that case, you don't want to average and smooth you want to keep those variables to the extent you can.
>> does that include gender? not only ethnicity but we all belong to at least 6 or 7 communities of identity. >> i think this is one of the those things you start and there will be some variables that-- the beauty of these things they call it data-driven design.
the data guides you. the data guides you step-by-step and it will tell you you know what? this variable actually is irrelevant. because it is the same across all your-- remove it. versus that variable shines!
so i think it's another one. >> pearl? >> thank you for those entertaining presentations. my question s so much of the focus is on people who already self identify into a group such as diabetes, such as heart disease, such as college age.
when we are looking at our million plus, how are we going to possibly have the context and the just in time information to keep them engaged with people who do not yet have a diagnosis or see themselves such a group? like cathy, if you did not have afib, would you be in the
healthy heart study? what would have gotten you there? i think that's one of our major problems. challenges. don't say problems. >> i probably wouldn't. because i live in an area where
i not heard about it. healthy heart has not made it to the agricultural area of so, because i chose to go ucsf, that is where i did. i since brought it back and have done several talks to administrators and teachers in my area.
because i think teachers are just a wealth of information. they are pretty compliant. they are going to do their thing and be part of a group. they are educated. after i became a heart patient, several of my colleagues have so, there in itself, what is it
about heart disease and administrators, educators, teachers? that was my favorite part. but i think we just need to advertise and get people involved. we need to bring it up to the conscious level.
and it wasn't brought to my level until i was actually sick. >> i think it's also important adds we talk about subgrouping people and defining them by their different demographics or communities, that we remember that choice and autonomy is pretty important in getting
people to engage. and so i have some concern that we could stereotype people by forcing them into their data pocket or disease entity. so this is again is marted of the reason why having some social networking capabilities around the system you're
building is important because it will let people align with other groups they identify with. it's important to give people that level of autonomy. >> how do you get someone to engage with this application or system? and your college students are
this great example? they are probably not going to be thinking about what their cardiac health will be like. but the solution isn't that maybe we have a top down like we are going to snit a room and come up with what do college students care about.
if there is a platform where you can have these emergent strategies and behaviors from groups that choose to group themselves together, and maybe it's the college students that are wanting to get better grades, or do boater tinder. we laugh because it is like it
doesn't seem like it's a serious health outcome but-- >> so you guys said far more eloquently what clustering is. so, you may have 10 kids who come from families that have chronic diseases, maybe diabetes, hyperademia, but they don't have it.
nobody ever told them they have their bmi is a little high. they are not going to come and self select into a group. they are going to select into a group that says, i'm concerned about my health. and you track that and you keep a variable history.
lighter if you decide to pull them into this group, and label them because they are prediabetic and their fasting glucose 130, but you don't start by saying, tell me you're fasting glucose i want to put you in bucket a. that's what cluster is.
you let them self select but you observe. because you should be observing in the background. >> that was a great point about having the right people in the room. when siri, the voice recognition software first came out, it
didn't recognize female voices. and when they did an autopsy of why, there were no female engineers in the room in the design. my question to you, who do we need be there in the room to collect the data we don't know we are missing?
that we don't know we are missing? >> citizen science. >> is it real and it goes to they with go shared governance. and unfortunately, the best place you can start is with your major demographic groups and the groups that you think will be
underrepresented. so you will have your highly-educated patients with chronic conditions. >> one of the big themes of this new initiative is to make sure that we don't repeat the mistakes of the past and i'll talk about it in my session but
out of all the funding that has been done in the last 20 years from the 95, nih, 95% focused on european populations. this is where you need your targeted sampling. you need to make sure the people that get left out are in the so for example, when we talk
about how my plan is to work the multicollege student cohort, we want a split of public and private, of minority-serving institutions, east coast and west coast, and we want administrators and students on the advisory board so that they are represented.
i think you need a shared governance system so that you make sure that the questions that get addressed are the ones that you're both majority and minority and the populations usually missing, are in the room being able to articulate them. >> number1 over the past
decade, as we started to incorporate beyond genetics, influences of the environment, the gut microbiome that makes an asian versus u.s. versus european, metabolically look we are learn figure we take the environment, the diet and the genome and look at the read out
of the system, which is the biochemistry, we are all each one of us have unique metabolic identity. i don't think we need to worry about making discrimination and stigma. we are all different. metabolically certainly we are.
the gut microbiome cannot be ignored. and number 3, when we put this all metabolic maps, men and women are different. if we try to correlate these metabolites, we are wired differently. how is it that genetics is the
combination of different things? there are traits and different patterns in how we are wired biochemically as men and women and as ethnic groups. so, i think the puzzle has to continue to evolve and gathering data, the biggest issue is to gain knowledge.
not only data but to integrate data so that we can link the genome and the metabolome and gut microbiome and gradually start to gain knowledge. >> so, bonnie raised some really important ideas on socialization but as someone who runs a cohort, i worry and we have an
upon i showed you but we haven't opened it up so participants can talk to participants yet. we think about social media and chatrooms and a lot of problems. what if an employer find out their employee has cancer. secondly imagine a scenario where a participant walks in and
says, thank you for my dermascopic lense and the $25 incentive and then within an hour you have 5-10,000 people saying what about my $25 incentive. i'm cures from what you have done, have you seen negative consequences to social
engagement? >> this was something that we were very worried about and i spent a lot of time talking about this particularly with the groups at the nih that has done this work. so nci has a lot of social media presence.
and i seen progression where initially we were very, very concerned about harm and miss information and how we were going to cope with that. and over time, i think we are starting to recognize that the population is smarter than we gave it credit for being and
they can do a lot more self correction. nonetheless, i think it is important for these spaces to be moderated. and there is a tough dance to do because when you have a moderator, step in. it can dispel.
it can shut down conversation. but, the moderator serves the function of if miss information is not corrected or the group doesn't police itself, there is an opportunity to act. this is also the issue of being alert to health emergencies. suicidal ideation for example,
is one. now i'm on a college campus so there is something that we can do about that. but i think those are some of the issues that need to be dealt >> we are down our last 5 minutes. please be brief.
>> a couple of things. on the digital communities of interest, there is also ways to build with the hierarchy so the people with the 12 dollars are only going to talk to people with the 25 dollars. you can control that in your space.
the comment i wanted to make was when we talked about bringing people in who didn't fit into a disease state and as a nonscientist in the room, the best i can say is i know from the black swan we are limited by what we know. how do we broaden that.
i'm cures if nih considered using any other kind of consumer tools to go out and target so for example, this may or may not be a good analogy. you can target people using social media as an ad platform who have already 23 and me. we know they are inclined to
find out what their genetic basis is. find out about their health and they care about that. is there any inclination to use consumer tools to market to people who aren't in a group already. >> i don't work for the nih but
if anyone wants to answer that question, go ahead. >> so i think the question is not really directed to nih but to the working group in terms of how are we going to recommend that people be solicited and invited to participate and i think we are in the midst of
those discussions and welcome all the input here today. >> marc? >> marc fletcher from healthy two quick questions. or points. one, obvious that patient leaders like cathy are incredible assets for studies
like this, like healthy heart. and i really encourage the committee to engage patients in thinking about the research questions and the designs at an early stage and they will be your advocates for patient engagement etcetera. the other is, i really want to
highlight what bonnie has brought up a couple of times. and i believe this is the next us of patient engagement, patients are so in to these they always think they are different than other people and they are. and they want to learn the
answer for themselves. mobile and personal technologies which enable these studies for a whole variety of different types of outcomes in the dense time series manner you need for these studies and precision medicine. what is are more precise than getting the answers for an
>> last audience question. >> andrew from the mount sinai one of the questions we have dealt with in our own work is a question of bias in terms of people who volunteer for if we are talking about trying to build a national cohort of a million or more, i'm interested
in how you think about trying to deal with that in terms of generalizability that results to the rest of the population. this is something we struggle with terribly. you get people who are engaged, like kathy, and many others who are excited about the prospect
of volunteering and feeling patriotic and contribution, how you generalize from that to the general population? >> i think it's also the thing that worries me about recruiting too extensively from places like 23 and me. if we only get the people that
are already converted, we won't understand some of the difficulties of only rolling the people who are hesitant about going there. >>> i have one more question that i was supposed to ask they have to ask and it has a lot of big words so pay attention.
what procedures would you recommend to test the interface and engagement functionality? and what use outcomes should be monitored? the last one before lunch. there are a lot of different techniques. the reality is you do-- the
best is if you can bring somebody into the lab and watch them using it and have them think allowed about what they are doing but that is a very small sub set em of people you can do that with. and some of my work with looking at gaming for things like
cognitive training, we have gone towards big in the wild sort of data collection where we don't know exactly what they were thinking and what they were doing but figuring out clever ways to instrument the application so that we can get more data about what they were
i thought of this earlier when rhoda was presenting, showing the trace of the movement of the pen. we have been looking at similar concepts for like instrumenting, like on-line designs, mobile apps, to try to get at a little more like maybe what their
thought process was, what they're interaction strategy was. so long story short, i think it ranges from very qualitative to really rich one-on-one interactions with users to just widely disseminated highly instrumented quantitative
analysis. >> one good thing is we know what most of the data look like and that we'll know very quickly if things are not looking good because most people stop using whatever the technology is, very and so, i think one of the other questions that needs to be
addressed is we talk about needing to refresh and re-innovate, but i don't think we have got good data on how often. and i think that is an important empirical question. >> so, if you're in baltimore, come to our office, we have a
wall of t.v. screens it's our own houston control center. so i can see every single function in my product and i can slice and dice who is using that function when, what regimen they are on, are they people on this or this or this gender or whatever.
i can see all of that and we actually use that to drive day-to-day product decisions. it's fascinating. we see people struggling with entering their meds. you make a med entry change and the number of med entries the next day triples so it's like i
did something right. i think in addition to welcoming them into room which i think should you do anyway because it creates affinity and gives them a chance to share ideas. look at the data coming at the granular level from your application and then embed tools
like google analytics into your product so i can tell every left turn and right turn they make in the product on an android device versus an iphone and ipad and i can start to see that kind of engagement and then i accident correlate that and cause 8 it back to the things i
want to observe so i think those are two other techniques to use. >> i will say really quick because i wanted to say more for example, we have been taking like a simple survey but if you have it controlled by a game mechanic rather than just clicking, you know more about
like maybe they almost answered this and then went over here and yes you can kind of do the stealth data collection i'm spero manson, member of the working committee. it's my pleasure to moderate this afternoon's panel which really is broken into two
subpanels. the overall topic focuses on variables better thought to be measurable via mobile technologies, and we've broken it into two sections. the first three presenters will share with us their experience, vision and recommendations
regarding this subject, in terms of the environment. each of them will have ten minutes to present, we'll then follow with 20 minutes of panel discussion as we did this morning and i'll be certain to protect the fact that we have at least 20 minutes for the
audience's participation. we'll then quickly shift to the second subpanel, which focuses on behavioral variables thought to be measurable via mobile and the first subpanel this afternoon we have dr. michael jerrett, my immediate right, professor and chair of the
department of environmental health sciences and director for the center for occupational and health at the ucla fielding school of public health, presenting on environmental data being collected using e health technologies, followed by dr. kevin patrick, professor at
university of california san diego and adjunct at san diego state, presenting and focusing on the collection of contextual data examples including matters of location, place and activity. that can be profitably acquired using mhealth technologies. and third, dr. jeff kaye is a
leighton professor of biology and medical engineering at oregon center and oregon health and science university, portland medical center. i rode with dr. kaye on the bus ride this morning and he promised to be provocative in his remarks, and i assured him i
would be supportive in that regard. so dr. jerrett, if you would begin. >> where is the clicker? >> your timer and your slides. now, how do we get to -- >> thank you, dr. jerrett. that was a lovely presentation.
>> it's funny to be at technology conferences where everybody is fumbling with the it's a pleasure to be here and share thoughts and how to use mhealth and mobile sensing to better assess environmental i've been given two tasks, one to tell you what's possible now
with mobile technologies for assessing environmental exposures and what's likely to be possible in the next three years as the precision medicine initiative rolls out. useful concept to organize our thoughts around the exposures we might want to measure is the
idea of the exposome, first coined by christopher wild, composed of every exposure to which an individual is subjected throughout their life course. we'll never be able to measure that you about i think it's a useful thing to think about what we might want to measure and
some challenges we might face. he breaks it into three spheres, the internal, we've heard about biomarkers, metabolism hormones. general external which kevin patrick is going to talk about, so social capitol, socioeconomic neighborhoods, and specific external like air and water
pollution and that sort of thing. i'm focus on the third realm of external exposures, and a useful organizing framework was developed by a committee i was on for the national academy of science, given the small task of plotting the future of exposure
science in the 21st century, and really what exposure science is about is coming to grips with what my colleague tom mccone at uc berkeley calls the exposure moment. you can see in the central box stressors, something that's good or bad for you, it could be
green space exposure that makes us relax, it could be air pollution exposure that affects asthma symptoms, it accumulates to where it could affect the human receptor, through the time activity prism that people live in this can bring the stressor in contact with the receptor and
produce a potentially meaningful biological dose. just outlining technologies typically use to assess external exposure, remote sensing, satellite imagery, geographic information systems, nanosensors which we already talked about this morning and i'll talk about
in a few minutes, participatory assessment, ecological, geo local and biomonitoring, and we look at the exposome and framework, the first takehome message is mobile technologies will rarely be used in large cohorts as primary means of acquiring environmental exposure
so we need to think toward an integrated framework where we can bring together information, we can get from mobile technologies with external measures of the environment to get a better assessment of overall exposure our participants are subjected to.
the two most critical components that we can get from cell phones or or mobile technologies are location and physical activity. we wanted to validate information from cell phones research grade quality for physical activity in geographic location.
we used the framework where we took model date a on environmental exposures. we oscillate the surface up and down by the hour to get a details assessment of what the exposure is likely to be at any given time take information from the smartphones, so for example
are when you're in a taxi you receive more, you're in the center line where it's concentrated, depending on what mode of transportation you're we use the physical activity data to better adjust the intake of pollutants, infer the breathing rate if we know the
person's weight. this results from the initial pilot study i'll show you. we have a larger study of 180 people that we're still analyzing the data, so i'll talk later, it's big data. so wee had them keep travel diaries, the gold standard for
transportation analysis, so they mark down every time they move from one location to another, there were three activity measurements so the phone with the software that we were interested in testing, the calfit and actigraph and bodymedia sensewear, another way
of getting at physical activity. this is just one day of activity of our participants showing their metabolic equivalent levels where the darker colors are showing more active, and then this is the air pollution exposure that we overlay on those points, and one thing that
jumps out at you very quickly, it's a rich data set with a lot of data, it's also messy. those clouds of low metabolic activity levels tend to be where people go into buildings. if we did it now we wouldn't know if we had gone out for lunch or were in here because we
get air from our gps, it's a messy data set to deal with. i draw your attention to the circle in the bottom of your right, to show that the respondents had 6% of their time in transit on average, which is maybe a little lower than in america, 8% here, but because
people in barcelona are walking, biking, walking to public transit and commuting in cars, exactly same time the pollution goes up, one small slice of the microenvironment accounts for 24% of inhaled dose. that's the kind of precision we can bring to exposure assessment
when we have the information on location and physical activity that we wouldn't get in we didn't have the cell phones. we just assume everybody is a breathing machine sitting on their front step and give them the same exposure. how accurate is the information
that we are able to get from the cell phone, from the calfit software which was developed by my colleague edmond seto, this shows some of the gps measurement we obtained over several days, it's of high quality. these are the calfit plots
showing physical activity levels throughout nine days, and then this is a comparison to the current gold standard that's used in most studies, actigraph, you can see for light and moderate physical activity it's virtually identical and we have some underestimation at high
levels of physical activity which we know how to correct. just to summarize, we did get very valid measures of physical activity that were comparable, the gold standards. gps data were of good quality but messy and hard to clean. the wear time is one issue, when
you're going forward with this cohort, we only got three days of good data compared to five days of good data with the current gold standard instruments because a lot of people forget to charges phones, they don't carry them, the others are easy to wear.
i don't think i realized this until i worked with data, data get big, very fast. this is having two sensors sensing every ten seconds, and we have something like 10,800,000 data points for one week for 180 subjects. if i scale that out to the
million subjects in this cohort, it's 60billion observations per week that have to be automated, cleaned. big data quickly. what about commercial apps? the most prevalent is a noise monitor, many noise metrics can be used.
we're finding in studies in europe we're getting very good data out of this application, i'm not advocating for this particular application, that will correlate well with gold stand and instruments but like most people carry cell phones in the pocket it doesn't work on
well if the microphone is not exposed. wear issues, identification. i had a little animation i was going to show you from my own trip recently to europe but it didn't work on the computer. suffice it to say i was being taken on a slow ride by a taxi
driver and it classified me as being on a bicycle. [ laughter ] there are problems there. and the other problem is the black box. you can't say it's replicable science because move says it is. it's an algorithm, a challenge
when we use commercial apps. sensor technologies are moving fast. we have radiation, air pollution exposures, fitness levels, being measured on externally worn instruments. this was designed by rod jones. we were able to get high quality
data but it took a treatment amount of effort. we're not there yet. we're moving toward instruments like that that can be put in the home to measure many things, that's something we want to think about because how many people wear it complicates
post-processing, when i go from inside to outside the humidity level changes and it takes the sensor a while to recalibrate, under the realm of the outwearable sensors but we're seeing the miniaturization of the sensing technology, from a company just down the road in
newark, california, spec sensors, measuring carbon monoxide, micro processors like this could be worn on a lapel with blue truth to the cell phone, nearly perfect research grade information, scaled up could be produced for less than a $100.
are essential for the linkage to model and measured environmental exposures, if i had to say give me only one thing, that's what i would want. other apps for noise, travel, exposures are showing promise but we need to do a lot to evaluate them before rolling
them out. there's going to be the problem of black box if it's a commercial application, and then direct micro sensors are not there yet, likely in the near term given the high level of activity in this technology space right now to have
breakthroughs that are still going to be useful in the time span for this cohort for wearable technologies and so-called wearable technologies when they are embedded into our homes, our cars, our public transit systems. so i'd like to thank you for
having me here today and i'm looking forward to the discussions over the next couple days and i'd like to thank all my funding agencies and to thank you for your time. >> it's great to be here, dave gustafson and i were talking, this is the panel after lunch so
hopefully people are awake, with luck. delighted to be here as well. thinking about teeing up this presentation, i never thought i would come back to this but i started my life as a family doctor and this article was published in "the lancet" when i
was in training and i went to direct the family medicine training program at the university of utah. the components of disease, of sickness are disease, illness and predicament. we were taught people don't come to the doctor because of
they come because of predicament. they can't climb the stairs, can't spend time with family, things that are valuable to if this initiative is going to be successful, it will deal more with predicaments than it will with disease.
even though of course we want to make sure that we're appropriate this way, and predicament is a complex of psychosocial issues which impinge on the individual and space, place and time are paramount. i have the opportunity to talk about the context of this space,
place and time. i'm going to talk about three things, i'll overlap with what mike talked about but also wearable cameras, image-based tracking and social influences. what about location tracking? we've jumped into this face under the gene environment
initiative almost a decade ago, i remember the grant in 2006, a personal activity, location measurement system. we have used over the years a variety of the types of devices that have been out there. one kind after another after another.
we were the only one funded in the portfolio that was actually looking at the notion of tracking merging gps data with location data. now this is really exploded. there are lots of different devices out there but i will say that if you actually want to get
good location tracking, even though you can get pretty good why the cell phone the battery burns out quickly, it's a limiting step to get precise as this initiative goes forward unless somebody solves that problem it's going to be a challenge.
for some studies in the cohort the notion of stand-alone gps trackers are still quite good as are the other methods by which one infers location from communication strategies and devices, it will be a hybrid and a blend of these things as we go along.
so again our platform fuses physical activity data with gps data and can get some level of intensity. as mike mentioned if you understand intensity you understand level of exposure to we can construct detailed pictures of a day, a week,
longer periods of time and get some understanding of what people might be exposed to over that period of time. and we specifically-- and this is also of interest potentially, we built this platform as a plate form many researchers can as of a few weeks ago we've had
147 users, 16 countries, data on over 16,000 participants, and imagine this if we roll up this kind of data across a variety of people in this cohort and do some comparison studies that mike just alluded to. we've done validation of this. several validation studies where
we validated some of the strategies we use to recognize transportation pattern, location, and other things. several papers came out of this first generation palms. this was just palms, an activity location measurement system 1.0, data on the left-hand side as
you can see bringing in the-- see, bringing in the data. point is different. they have to be essentially brought together. those also have to be included with surveys and other sorts of you'll hear later. and i gave credit to my
colleagues, jacqueline kerr and marta jankowska, coming up with this framework related to how we address the data-related issues on the one hand, bring them in, infuse them to begin to understand the meaning from these data, and then push them out on the other side to
understand exposure through time, through space and as a function of behaviors. so this is really-- this is a non-trivial process, a big data process in many respects. what's the challenge here? we've always thought about access as being on the left-hand
side, you see buffers as the crow flier, or 500meters, or 1000meters or whatever people might be exposed to things but that's actually not a good a measure of exposure is where people travel through time and understanding where people travel through time and space is
really critical. and you can see the statistical and data challenges when you begin to get more and more and more granular over time from just a few people capturing data on a weekly basis to more people capturing more data on a minute by minute or epic by epic basis.
it becomes pretty compelling. so we have again-- we're digging into this trying to these out methodologies that are this is the data platform that hopefully might be informative for this particular project. i wanted to touch on wearable cameras.
we haven't talked much about i'm not sure i heard anything talk about this but cameras are exploding in our environment. we know this. we're carrying them in our pockets in our phones. they are ubiquitous. cost and size is dropping,
widespread use, go-pro, who thought a company like go-pro who explode. around visual analytics, big data revolution, scientists around the country and world dealing with these data. note here this comes there the mobile data to knowledge
project, this is part of this new nih-funded initiative that santosh kumar is leading. our work looking at this about gps and accelerometers, enhancing this with wearable camera data, can tell you what's happening in the moments. all of us who do physical
activity research wonders what happens when the accelerometers go silent. you can augment that. this was by senscam. we're developing in the context, a group at university of massachusetts, mobile data to knowledge initiative, low power
wearable eye tracker that tracks both forward and tracks the eye itself. i know we're in silicon valley, everybody remembers challenges with google glass, that was for consumers but this is for researchers, that might be important to be nested in this
initiative for this setting, this is going to be available within the next 2 to 4 years, the lifetime of this project, and we're already prototyping for the group at georgia tech and massachusetts, our use cases on this relate to smoking and to-- this is going to be used
in the craving and smoking side. the goal with this is to measure visual exposure through these wearable cameras, for example identify whether a person is watching television and not only whether they are watching and thus likely sedentary but what are they watching?
if television is bad for you, if we know television is bad for folks, why don't we try to find out what's on tv and what might be damaging people? with the ubiquitous screen environment, it might be good to this will be online at the time this cohort is fully stood up.
finally social influences, i wanted to build. bonnie did a terrific jog in terms of engagement and work she's been working on but i wanted to focus on one angle. there are lots of examples of use of social media to understand what's happening with
surveillance of things and twitter analyses. if it's important to understand the trajectory of health of someone, upwards or downwards, we can think about using social media for particular online talks is what people think, availability of content
and who people are connected this precision medicine initiative is going to be about individuals and also the social networks, two people share this information with and how they connect with others. this is just an example. this is a work that we just
completed within a two-year intervention of college students for obesity, and this is-- we've developed a language, my ph.d. student gina merchant came up with a healthy lifestyle language and we're tracking over time in those in intervention and control group, finding
there's a divergence, a positive greater use of this within the people in the intervention group and that's a way to potentially track what's happening with folks. so monitoring social media data, we should be able to get these data as well from the people
participating in it this cohort. i would add we talked about the framingham study this morning but my colleague james fowler said the robert wood johnson foundation funded them to add facebook data to the framingham study, now something additional collected, you heard this
morning about what a great job that group is doing on this. we'll now have facebook data, additionally, to this. i asked james about the million person cohort, you're going to have interesting connections between those people in the cohort and people outside the
cohort, very interesting things to think about working through as far as analyses. key issues, we'll talk about these things, we're all dealings with issues related to privacy and consent, privacies are really key, hardly anything is more specific to you than your
gps trace during any given day, week or month. and then others, imagine camera data or social media, other people who may or may not be consented directly into the there's a good group working on this initiative. we're working on in it robert
wood johnson project. in conclusion, contextual data can support the precision medicine initiative efforts to improve how disease, illness and predicament are addressed. i look forward to talking with you about this. [applause]
>> good afternoon. i was asked to speak about data acquired from the home, and i want to change that to home-based data, and then i thought i might change that to home as a hub, because that's really think we end up being, i'll explain that more in a
but first i want to go back to our challenge with you michael already looked at, to provide important data or the variables that would inform the pmi. i think this really is about quantitating phenotypes, it could be called the human phenoproject, and there's
another piece to that which i think might be called the epiphenome, and that's the effect of the environment, treatments, exposures on this quantitative phenotype that we're trying to get at. facilitated by technologies. the other part is really what's
ready now, what kinds of technologies are basically are there valid and robust enough to be part. part is important. i don't think anybody is going to have a single solution for every million persons or more. i think that we have to think of
this as more of a human phenome project where we quantitating diverse phenotypes. they have to be validated, and sensitive, and i think we need to make sure that this is as difficult as the human genome project was, and it's not simply putting an actigraph on a wrist
or opening up and app. people realize that. to conceptualize data, on the x axis is the four vs, the fifth is value. on the x axis is volume from small to large, data size, on the y is velocity from frequent measurement to infrequent.
the color scale on the left denotes variety of data from alike to diverse and the size of the circle indicates the certainty of the data. to draw your attention to the bottom right, the red circle where genomic data is, it's clearly considered to be large.
it's not collected very frequently, it's of high potential variety and we think it's pretty certain. on the other side of the x axis there is demographic data, which is on the small side. it's infrequently measured. it's probably fairly alike in
some ways, and fairly certain. i really want to call attention to these three boxes on the top. these are the kinds of more continuous or more frequently captured data that is where i think the develop of the technology approaches that have been described already today and
subsequently lie. up on the upper right corner is our everyday behavior or activity data. the ef is environmental factors kinds of data. small yellow circle is the ecologic data. with that in mind, there's lots
of ways of approaching data. by type, frequency, certainty, method. we have to be careful not to fall back on the law of the hammer, so everything could have a hammer, everything is a nail. if you have a smartphone everything is an app, use your
analogy as you like, but really think about this conceptualization as the internet of things, a connected world, with less focus actually on the methods and devices and more on when, where and how the data is going to be captured. and so with that in mind i think
that the-- one of the organizing locuses of capture and actually something we'll facilitate integrating this data is the home as a hub. and so the home is, as we all know, we spend 1/3 of our lives asleep, and in conferences as well as at home.
but there's other important activities such as going to the bathroom, meals, social engagement, entertainment. we have no idea quantitatively about those, and they have a greater impact than those we think about in classic bioresearch.
it's where you are when you're sick, if you're not in the hospital or emergency room or zoom care or wherever. it's consistent with the current ideas about bringing health care into the home. i think that the populations that are driving the majority of
health care expenditures are home anchored. the old 5/50 issue, 5% takes 50% of resources, people with chronic disease, chronic illness, chronic pain, 100million people, a shout out for alzheimer's disease, the most expensive disease in
america, these individuals are not all in nursing homes. most are at home. there are the special populations whose home is not necessarily the classic, the homeless, prisoners, transients and we already heard about dormitory residents.
what technologies could be placed in the home? i'll describe our approach over ten years to creating a life laboratory. this is taking a laboratory as the person-- the person's home as a laboratory, and creating a scalable platform where one can
be technology agnostic but collect the basic fundamental elements of data that you might need. so up in the upper left corner we have been using passive ir sensing and more recently beacon or blue tooth technology with a wearable to collect activity by
location and sleep and other kinds of ideas. contact sensors on doors can tell you if somebody is in and refrigerator or home. physiologic sensing will tell you something about body weight or pulse or actually in this case the temperature of the
bathroom, type of environment. med tracking tells you the time of day when you open the lid, a key feature of data, one would hopefully want to capture. we monitor phone use in different ways, we don't listen in on phone calls. we leave that to the nsa.
but it is an important measure of social activity. the computer itself, it could be a lap top, pc, tablet, a wonderful sensing platform where you can get basic data about a person's psychomotor abilities as well as using it as a platform for experience sampling
or pushing surveys more frequently than usual. driving is based at home where most trips are starting and ending hopefully. that's something we've been monitoring. finally it's important to keep in mind that any system should
be able to plug and play because new technology comes along, and new ideas about sensing things, and so this system has been designed basically to be able to plug in based on common standard communication protocols, those new and evolving technologies. this gets scaled out into the
community and the data comes there are four principles that are difficult or important actually i think in doing this kind of work. first of all, understanding the user, the user is not just the volunteer but it's actually the research teams that are in the
field, we shouldn't ignore those kids, but once you understand that problem then we take the technology into a smart home or smart apartment. many universities have these. we have one. ultimately it never works as you think it will in these
environments. one would need what i call pods of life labs to really understand how things work in the real world and then successively scale these out. in parallel the data analysis aggregation, development of algorithms, is not to be
underestimated. that was brought up by colleagues here and we need to work a lot on that. in closing i have three challenges to collecting this kind of data, first is visualization, most of this data is coming in frequently, it's
unsupervised. we need better ways to visualize this is a spiral plot showing activity, noon, midnight, successive circles are weeks. this pattern, this person, which is typical, is going to sleep and get up at a typical time and has typical pattern of activity,
but as you follow this person over a couple years in the middle you see this person has developed parkinson's disease, there's clearly a change pattern, and then if you look to the right a year later this person is treated with standard therapy, and that pattern has
normalized. second challenge is-- you need to plug and play over change. people are still using land lines, they use mixed environments, land lines, smart phones, cell phones, and so to collect this kind of data is very important to be able to
think through that crosstalk across the technologies. and finally validation. so all this unsupervised activity data in particular, it tells you things that seem to be valid, have face validity, but we need to think more about biologically or how increase
validity. we show cognitive function declines or actually computer use declines in people with mild cognitive impairment at home. mri scans show atrophy in the medial temporal region, the typical place alzheimer's is thought to start.
i want to remind you technology moves quickly. on the left is the ski lift in 1956 on mount hood. it took 20 minutes to get to the top and it only ran half the the other side is the tram which takes three minutes to go from the bottom of the campus to the
top, millions of people have traveled on it, i'll probably jinx things but it seems to never break down. >> we'll thank each of you for such informative presentations. i'm going to jump start the conversation by playing ed mcmahon to your johnny carson
carnac. i'd like you to-- exactly. think ahead three to five years from now when there's been assembled the cohort of a million or more individuals and then through this particular which technologies do you think now and what you would project
forward in the best three years are best suited for assessing environmental factors thought presently to be critical to increasing risk of the onset of a particular disease disorder related condition, and i'm interested in what-- how you see those then aligning.
what's the sweet spot from what you now know and think will be the race as it may apply to particular conditions, and as importantly once you make that commitment, i would be interested in you sharing the logical path by which you got >> i was going to say there was
a phrase you used about, you know, which people-- i think that the issue that's really important is we really need to talk about what are the outcomes you're trying to capture. and that really has to drive the kinds of technologies that you're going to use or not use.
so if it's-- we heard some wonderful presentations this morning, looking at going, you know, what would college-age kids do, that's very different than what a population of people with alzheimer's would do. i do think that the fundamental-- there's some
fundamental things that were brought up. michael started with i think that activity and location are probably the most fundamental and currently the most fungible quantities we can look at, whether using the-- because the accelerometer is the
accelerometers is the accelerometer, that's probably the shorter term type of sensing that will be done but ultimately also interesting that we spend so much time in this conference talking about engagement. i think we don't want to engage at some level.
in other words, we want to be able for some studies, not interventions but understand human health and behavior, in the background like driving your car, the computer is collecting amounts of data when you want it or your mechanic wants it. >> or your insurance company.
>> if you agree. >> that's one paradigm. that's another use case. i'll stop there. >> i would second the notion of activity. a decade ago we talked about measuring in the context of space no one dreamed we would
have people using accelerometers. we completed a study of the fit bit hr, it's as valid as accelerometers for measuring sleep in young people, a critical thing. when you have this ubiquitous presence of these consumer-grade
technologies that's going to help. that's a big part of the pathway. when you know activity, you know other things related to exposure. we're not as far along on the assessment of some of these
environmental stressors in terms of highly portable and highly low cost times of devices. it's just going to take a few more years. i think we're going to get there within a few more years but in our research we're not there now the way we'd like to so i
think-- i'll pass to michael, it's a smaller subset where we'll be experimenting with the methods by which we'll do that i with clunkier technologies to factor that into the larger cohort as time goes by. >> i would like to comment to my colleagues.
i think the idea of the home as a hub is a good one because there are instruments we can deploy now that are inexpensive that we can get high quality information on environmental conditions for a place that most people spend 70% of their time. that's considerably less
complicated to do than trying to measure on the person as they are moving. if we do know where they are, we know that with some certainty, then we know exposures they are facing in their home for air pollution, for example, electromagnetic radiation,
potentially for noise, these are factors that can really greatly inform our environmental if we look out at say three more years, certainly with the activity going on i think we'll have wearable sensors for air pollution, for noise, that are quite miniaturized.
we may want to look at how you analyze data as people pull cell phones out of their pockets. there's a lot of information that can be gleaned. there's a lot of sensors, they have accelerometers, gps, and they have gyroscopes, they have microphones, they have light
sensors, so all of those can be used and it's going to be a matter of thinking about the analytics that we need when we capture very short moments of time, and then we're quickly moving not necessarily for health interests but an area where we'll have embedded
sensors, whether it's three years or five, i'm working on a project funded by nih in the imperial valley where they don't just the government. google is trying to embed in cars, transit, areas in the city. that has the advantage we get
ongoing continuous information so again if we know the location of the people and have high resolution measurements of environmental data for things we know matter, temperature, light, air pollution, noise, electromagnetic radiation, we can rapidly assign that,
something that will evolve. the big challenge so far is we don't have-- i'm heartened to see people from the private sector here but we don't have collaborations yet where the private sector is sharing information to use on a regular basis, that's one of the
challenges we need to confront, no way academics can compete with resources of google and intel, but there can be collaborations, for the researchers they are getting mass information on movement and cell phones and google and other companies like that have already
done much more research than kevin and i can muster even if we're well resourced on how to clean data, how to deal with geez gigantic data, they have the backbone and infrastructure and we need to foster collaborations, i hope that's something that will come out of
>> one final thing, i did want to emphasize the notion, assuming we're successful in the shared governance, handling the voluntary contribution of data, there's been a lot of discussion about this than at the first meeting, we can get people comfortable with the notion
we'll value these data and handle them with the greatest level of respect, the signals we can detect from contributed social media data, from contributed other data people can provide us will be very we can discern the signals in those data about what's
happening with folks related to what might be making them sick or what might be affecting them. that's really critical. the group here dealing with consent and with understanding the meaning for being a participant can't be more important than the data we're
talking about because of the level of sensitivity that they have. >> michael, the one slide you showed with respect to gps and location, you can't help but come away with being struck by the mass of the data. and it sounds like we're
addressing the data management analytic and computational challenges not withstanding the complexities. where are we at in translating the findings, observations, into meaningful things to inform decision making? how do we think about that?
jeff, i think that's also true with respect to the massive data your life labs probably are acquiring. >> so i don't want to make excuses but i think actually that much of this, the science of this activity is quite early. when the human genome was
sequenced, there was amazing potential, but it's only now that we're actually understanding where that potential is. this kind of data collection applied to clinical trials, there's this massive backlog of potential treatments and the
methodologies for clinical trials are still using the same methodologies that james london used in 1774 to look at sailors treated with scurvy. i'm overstating a little bit but really we could actually do trials faster with smaller sample sizes if we apply these
kinds of technology. there's a lot of public health opportunities for the environmental sensing that we are going to see very soon. but it's still-- i still think it's actually relatively early in the science of this kind of >> totally agree.
i think this is where this initiative has an opportunity to crosswalk with the big data to knowledge initiative, the mobile data to knowledge. there's several more years of that to be done. my most interesting meetings now are with data scientists helping
us deal with multi-level, multi-careful, different aspects of data, and again it's a good way to say it, we're in the early stages of this, developing new methods. the other thing i point out is opening this up. one thing we're doing with the
nsf project is not us coming up with answers but we're trying to create a buffet of the kinds of data available and then making it more available to many people to be the citizen scientists or people like us or non-profit organizations two-- who have an interest in this, in effect
getting everybody working on the project, but it's only doable in the sense of precision medicine initiative cohort if we're able to gain the trust of participants to open as much. there will be a distribution, some people will be happy giving everything, some people only
this but not that, but that's that's what's exciting is getting this hopeness to this to have other folks working on it. >> yeah, i think ideally we want to be able to convey the information to the individuals, especially about environmental exposures, in a way that informs
their everyday lives, so that they have a tremendous incentive to stay committed to the project and to the cohort because they are gaining access to better information when they want to go out for a jog, they can know where the pollution is. they are gaining better access
to where to put their children in school, because they don't want to be a by a highly polluted area. those are tricky because data out of sensors is as i showed quite messy and quite uncertain. i think the science of how we communicate this imperfect data
to people on an ongoing basis hopefully through cell phones, i've been involved in european union initiative that is called city sense, so its whole intent is using technology to empower citizens to better know environmental and other risks in their daily lives.
and that's the biggest question that we've come up against is how to convey the information that's potentially useful to them without unduly alarming living in los angeles, there are going to be days we're above government standard for air pollution.
what do we tell the participants? to curtail your normal activities? if you look at the times of indices developed by traditional government agencies they are not very clear on what to do, and how you should respond to that
so when you get more of this information and it's very uncertain, this is an area of research that we need to work on to try to better convey those uncertainty data about risks to our participants. i agree with my colleagues we're still sort of grappling with
i do hope that we can have some studies that are part of this initiative and others that are going to be of the size and quality of the ones i've been engaged in in the european union because i don't just go over there because i like the social climate, or like spending 15
hours on a plane. i'm going because they are doing work not being done in north america and it's something we need to think about as we go forward in this cohort, how to play catch-up because they have invested a lot in characterizing exposome, characterizing citizen
science, that same level of investment is not forthcoming yet, with all due respect to other great things done at nih. your candor is appreciated. let's hope up to the committee members in the audience. >> it was really terrific. i'm impressed that thinking
about environmental exposure one of the major areas that we don't seem to have a good handle on is diet, and everybody believes that variation in diet has tremendous impact both on our risk of obesity and many of the attendant diseases that result from that.
we certainly have a handle on being able to measure activity, and it would seem if we had the grasp on a good way to measure diet without people writing down everything they eat, even nhanes is for two days, that's not recognized as not being complete, i wonder about your
thoughts, thinking about the camera technology, jeff, that you mentioned, prospects for passive measures of diet. >> so first of all, cameras are tricky, particularly in the moment space. in most surveys of populations about attitudes and beliefs
about technology to bring into the home, direct visualization is not favored, except under very special circumstances. so most people don't want a camera in their bedroom or bathroom, even if you tell them that, oh, we're just going to look at decomposed images, it's
still not great. so i think that that kind of visualization may not be at least currently the way to go. i do think however there are ways to inch toward getting better data about eating activities, so i wouldn't call it that we have diet, but
certainly i didn't have time to show this but you can do thermal sensing in a kitchen, and get a fairly good fingerprint of how much time a person is engaged in relevant activities in, you know, preparing a meal and eating a medical. there are other kinds of sensors
that can give you that kind of doesn't get to what i think you may be wanting, you know, the elemental composition. person's diet and how well it's absorbed in their bloodstream but in fact one could consider that there are point much care kinds of technologies where one
could obtain many analytes in the blood at home, and i think those will develop and so measuring nutrients on a regular basis in a home environment has a lot of potential and we probably will see something like >> the project is looking at dietary measurement were part of
this portfolio the group was funded in, and i think they had the hardest time, the notion of diet, everybody knows it's difficult. three were focused on image-based assessment of wood working through the issues, obscuring data in the
background. this is going to continue to be a challenge. i've come to the conclusion it's going to be a set of things ranging from some of the-- i like the notion of the times of activities, eating at bedtime, overeating, other things which
might be inferred from wearable sensors or other things by virtue of how long eating episodes are, what time, that don't require a nutrient intake but there will be community level things. we know the terrific work being done with respect to community
availability of foods, precise locations, are they or are they not in fast food joints, are they or are they not in healthy food locations, and then additional data at a macro level. some mathematical solutions will get richer to the extent we have
more data to feet and give them to bright mathematicians. we'll be able to understand this better from this variety of data sources. >> from the discussion about the importance of the home, should we as looking at the one million
people going forward be looking at family units as opposed to individuals? >> yeah, i i think that's a very good analysis. it's very difficult to define the families. i've been involved in several studies where we've looked at
parent/child pairs, and that's been informative because you can then look at the interplay between, say, physical activity when the parents are there or not there, and start to understand some social dynamics indirectly at least. yeah, i would say that's
probably-- now, there are many families that don't live in the same home so that raises other challenges, but in terms of social support as well and potential contacts, that's certainly a good unit to think >> i happen to have been in a meeting this last week with a
local epidemiologist tina chambers and another, andre mcroy, cross-walking what this initiative might become with lifespan challenge also coming out in ctsas and others. actually not only potentially families, but test bits, and places-- you had a picture of
homes and it doesn't have to be every home, just enough homes where you might have everybody in the home participate in some way so you're getting an understanding. we're all talking about the anchoring of data to place. it's critical to anchor data to
place where people are at home, where they work, where they travel. southern california, we're the most toxic place along a freeway, people spend 45 minutes each day going to and from work. if you don't want to listen to exposure biologists tell you
what's on the freeways, it's scary. there should be serious discussion of at least a few times of environments where in fact we look at the concentrated microenvironments of families. again, we talked about social networks, by affinity groups or
others, that's important but the tightest social network is the family for good or worst, depending upon what happens in so i think some serious consideration needs to be given to some methodologies that would allow us to do that because that would then tease out some very
interesting dynamics related not only within family but i restate, place-based research because if we just enroll people and don't pay attention over time to where they lived for a long period of time, we will miss a great opportunity. >> i want to quickly make the
comment there's lots of research that has shown the health outcomes of individual person is highly determined by the person who is helping them, and the health of that person helping them is at much greater risk of declining by themselves, and so that interaction is very
underexplored and probably has a huge impact for discovery and understanding about the basic trajectories of change with illness. >> i have a very practical it's a tech support question but it's-- a number much you talked about "the platform" and, jeff,
you spoke about it. for us, imagining scaling to a million people, there's a continuum of, hey, nih with industry builds out a platform that is the study platform, it's the, i don't know, to make an extreme, the lockdown set of capabilities in the home at any
one point in time, and the specific cell phone. on the other end of the continuum is byod, you know, anything goes, practically when you guys are doing your studies today help us know what is the quote/unquote platform and that balance between byod device, and
stuff that can be controlled where you can control the security and privacy and maybe get more valid measures. what's that balance and what do you think we ought to be aiming for? >> i think the balance is that initially you want to actually
have things a little bit more prescribed and controlled, and so obviously you have to give up something for that. i think the sweet spot hopefully is i look at it as the communication protocols are actually standardized,so whether it's bluetooth, cellular, wi-fi,
x-10, those all have standards so as long as the platform allows you to plug those communication standards into a network, a home network, and bring that data out, i think that's the level of thinking about a platform, and what i've seen is it isn't very hard even
for us highly underfunded academic researchers to take, you know, a commercial scale or device as long as it has those standards, reverse hack it and get data out. if we can do that, i think with the support, the wonderful support of intel and google,
that platform could be scaled out easily. one other quick comment, there's this-- we've done this, this concept for better or worse sensors in a box, so the ability-- and there's some commercial products now that have gone this way, although i'm
not sure how effective they are, but the idea you could mail somebody a box of sensors, color coded, this goes into your bedroom and connects together is very doable, it's doable for actually per house $1000 to $5000, the same amount of money you could get that data
as well. >> i think it's a great having been involved for the last several years in one nsf and two nih functions where we built data capturing types of stuff we controlled this on our own. we weren't using cloud-based
services, now amazon. we have that ability. we even have the san diego supercomputer in our back yard, they can satisfy some of our needs, others they can't. i'm in the position of challenging our software developers, saying why are we
developing these in different ways, why don't we develop a unified approach? we don't have an answer. this is the way it happens, underfunded academic areas, but i think it's a critical question and we'll have to strike this balance of entrepreneurial
activities that allow things to happen and highly creative ways that we can't yet anticipate coupled with getting some sense of pulling together the variety of activities that are happening so we get more into standards, comparability, data standard sharing.
there's a session tomorrow morning talking about this on a panel, but terrific question and i think part is engaging researchers like us to think about what the requirements for that will be. >> i want to go back to the issue issue of measuring
nutrition or diet. this session has two parts. one is on behavior, which is what i think most of the answers were about, but there is also an international effort to measure what's available, and there's a lot of digital data from food companies and supermarkets about
what's available. some is being used to evaluate food company adherence to their pledges to change what's there, but i think there is a private sector issue with getting that i would love to hear your ideas about how we could actually measure the equivalent of air
quality in terms of-- what location to we get to the point where we're over the top with being able to make a healthy choice? >> another really good question. i worked with the environmental science research institute, geospatial business analyst, and
they do have marketing data that's highly resolved, and gives you a very good assessment of the types of consumer patterns you're seeing. you can see some of the areas of the city where clearly there are preferences for more vegetables or they are not eating dairy.
but the problem is that the information is coming in units that we can't compare to any census data. we cannot-- or to our cohorts. it's coming in strange marketing units they use called tapestries that are meant to segment the market but they are not very
useful for doing health so that information can potentially be incredibly powerful, when i've looked the spatial patterns are what we might expect with people with lower socioeconomics are eating more sugars, more fats, less vegetables, falling in line with
our prior hypothesis and what we know from studies but we've not been given access to the data at a highly resolved level so we can do these, as kevin is calling them, place-based studies to understand and link to the cohort in a meaningful way.
>> i was talking about not what the people in those communities are eating but what is in those communities to choose from, which is a different issue because of the marketing patterns. >> this is what they are buying, so it's something in between
eating and -- >> it's different, you can't tell from that. >> excellent question. i tried to allude to that. we had an initiative with the vons data, for a period of time-- or safeway, terabytes of data, the blue sky initiative,
kaplan was the chairman, looking at transfats and other things. if there's a way to get to the large food distribution companies, and i would say we have an opportunity to smile, lean a lot and encourage them, it's in the public interest to make these data available in one
way or another, that's what i meant if we add that in and some of the fine work that jim salas and others have been doing related to understanding data, we can begin to extrapolate and get a better understanding about what might be available in certain communities, that's an
overlay, much the same as any other environmental characteristic overlay that then one using that to make inferences about other things. i hope these would be additional forms of data and nest questions about precision medicine in. >> we have five minutes left.
i want to try and get in three questions if we can. >> building on things that are hard to do, what do you consider innovative stuff you're working hard on, in multiple research labs, but should be a commodity, if we have intel, samsung, bd2k, what do we need to do from
infrastructure? richard and myself and josh are wish we go could do patient linking, get access to medicare data, that's the refrain of ehr what is it from location or what is the one thing we need to not make proprietary and enable innovation as opposed to
commodity work? >> i want to understand the what would be the most innovative private sector contribution? >> what is the thing you're spending energy on that you wish you didn't have to spend energy on and lots of other people are
spending energy on, this is easy if we get these guys to agree, buy it, make it free and you could foes-- focus on innovative stuff. >> a big distraction is the processing of huge data, trying to align that so you're able to tell indoors versus outdoor
activities, you're able to-- somebody's walking down a densely on populated in street in san francisco you see they are not jumping inside the wells fargo building and jumping out every ten seconds, that takes a treatment amount of time. i was advising on a study in
paris last week where they are trying to get high quality information on people's commuting patterns and the only way they can do it is to have people go in and manually correct the data, and you're talking about four or five person years of data over three
years to just have somebody doing incredibly tedious tasks, so if that could be solved, so we could get the level of information on google maps out of our health data that would be a big step forward; >> as a clinical researcher, even worse according to jeffrey
a neanderthal trialist, this is preempting my question but three points, and i'm interested in your reaction. hearing this and everything else today, i'm imagining the biggest garbage dump in the world, imagining all the garbage and trash, imagine you need to sort
out what needs to be recycled and sent elsewhere or burned or whatever you want to do, there are holes in the data, every precept of drawing inference seems to be violated by problems with samples, missing data, incorrect data. even worse, or maybe better,
when most people hear precision medicine they are thinking we're just not observing things, we're actually going to give advice about what to do. we have to understand causal inference and realize over half of observational observations turn out to be the inverse of
what the answer is when you do an intervention because confounding and bias are hard to pick up and you're lamenting underfunded academic saying we have to depend on google and intel to do it because we can't analyze our own data. you say five years to clean up
one piece of data, curation and date a janitorship as the most important thing. if you look at the million person study and expected advice to people, what's a way out of the box? we can't just collect a bunch of garbage and hope for good things
to happen, can we? >> other than that you're enthusiastic. >> it's always the fda guy. it's always the fda guy. >> so coming from oregon where we recycle everything-- i actually think that as bad as we might have given the
impression that the data is, it's orders of magnitude better than self report data that's sparsely collect and totally unverifiable often, so i think really the first step is probably to focus on smaller, you know, not every million person will be doing the same
thing but probably having smaller cohorts with the full metal jacket of things we can do and moving those out into the larger cohorts over time. clinical trials, which hasn't been a cohort that's talked about, is a perfect set of cohorts, thousands of people are
enrolled every day into clinical trials, they makes commitments for years on end, there's a lot of structure to those, to the crappy old-fashioned data even, so i think there's-- those are the first opportunities, and i hope we didn't overstate our-- i'm quite optimistic.
>> you've got 15 seconds. >> i thought we were talking about advances but we emphasized the negative. i would say the other thing that you can do with this type of cohort is focus on natural interventions, so when new parks are put in, when the environment
changes, you're tracking individuals and you can see abrupt change in exposure that offers the opportunity that's not an exact clinical trial but something that's going to approach that. if we have the high quality data we can then move forward with
those comparisons that will get rid of bias and confounding you're mentioned typical in observational studies. i know you've been waiting patiently, can you think about wedging in your question into the next subpanel? that would be great.
i promise i'll come to you first. thank you, very, very much. we'll have the next three dr. ertin, dr. campbell, dr. stone. >> our previous subpanel inevitably found its way into discussing behavior.
and at a number of occasion person-environment interactions this particular subpanel is charged with focusing specifically on behavior. with the same task of providing insights into those aspects of behavior that are deemed to most measurable presently by mobile
we have three speakers again, dr. emre ertin, research associate professor in the department of electrical and computer engineering, investigators in the david heart and lung research institute at ohio state university. he will be presenting on
wearable physiologic sensors. we're joined by dr. andrew campbell, professor of computer science and co-director of dart net lab at dartmouth college. dr. campbell will be presenting on passive measurements of behavior using mhealth and dr. arthur stone, professor
of psychology, director of the dornsif center for self report science at the university of southern california, he will be presenting on self-reported data using ml technologies, we'll use the same format. please. >> and the clicker is --
>> it should be right there, yes. >> okay, thank you. i would like to thank the organizers for inviting me and enabling me to be part of this exciting workshop. so i'm going to be talking about the wearable sensors for
measuring physiology, or mobile health as it says here. we're all here because we believe if we can measure physiology of subjects into daily life routine, that we can understand the root causes of complex disease and risky behavior.
and hopefully enable large scale we all want our sensors to provide clinically relevant measures but at the same time we don't want our cohort to stick out in the general population. like the one on the left. so as the subject of physiological sensing in the
context of behavioral sensing, perhaps it's good to understand what we mean by that. the idea of behavioral sensing from physiology is if we can observe physiology, the different facets of physiology, of subjects, by surrounding them with different wearable sensors
perhaps we can see the underlying processes of psychosocial process that drives on the left we see all sorts of measures that we collect to give activity measures and then on the top the green is the activation of the system that relates to the stress so we can
see in this particular lab study, we can see that we can process the physiology to provide a signal that is at least correlating with stress on the left. we don't want our subjects to stick out in the general population which means wearable
sensors have to be required to obey stricter restrictions from power, size, weight, as compared to the traditional sensors in with those restrictions comes the limitations of those sensors and data they provide. so the system modulates several biological process, and now we
have essentially a sensor that looks into each of these biological processes with wearable sensors so in specifically we have cardiovascular monitors, motion, galvanic skin response, eeg, spirometry, temperature sensors, and we also have ability to
analyze glucose lactate in a form factor that goes to wireless sensing. for the most part sensors are just the traditional sensors put into the compact framework of together with limitations that comes with that. so this is how i see the mobile
health, or wearable sensor landscape. so we have sensors that provide specific information about specific pathways but they are somewhat dorky. and then we have the right sensors that are blend into the life routine, but as a result
provide generic information about motion, some heart rate information and things like in order to be successful in this effort, we need to move into direction of the arrow, provide sensors that provide both specific information about specific pathways as well as
blend into the daily life routine. we all agree we want sensors to provide clinical validated information about psychosocial process, giving some contextual clues. the sensors need to be continuously monitoring so we
can't have sensors that come out once in a while because the event of interest might happen anytime in the time so if you're looking for stress research, stress event can come anytime in your daily life. they have to be robust and i'm going to go back to this factor
so they have to have predictable performance in the field, and we hope that these sensors are not just engineers dreaming in the lab but informed and rooted in so as i said, most of the sensors we were looking were just taking the sensor and putting them into the compact
wireless factor. we have novel ideas specially suited for mobile or wearable sensing. we're all embedded in wireless signals right here. i looked up, there were at least 25 hot spots in the field, so we're certainly bombarded with
wireless sensing am the wireless these signals penetrate into our bodies and reflect and provide information about how our body functions. on the left is a sensor that is close to my heart. no, really, it's showing my picture of my heart.
from wireless sensing that can monitoring heart rate and respiratory rhythm. on the right, understanding our bodies are not flat surfaces, so why are sensors flat, trying to do skin conforming sensors that blend organic materials are electronic, most of them are
from science, different issues of science, and we have the contact lens sensor developed by a local company here. so we understand that, yes, the sensors are coming in the next five years, perhaps can make an impact, but before we can use these sensors we need to assess
accuracy and how they link to the outcomes that we care. just to understand the challenges of doing that before we adopt these sensors i'm going to switch to sensors that are much more developed, cardio monitors, and try to sort of highlight some things that are
so not every heart sensor is the same, basically the same message. when we look at cardio monitoring people are used to looking at ekg wave form, when you say heart rate it's basically how peak to peak changes across time.
so when we have now sensors that are built into the the i-watch and microsoft pen what they are envisioning is average across one minute, perhaps giving heart rate information and activity but not variability, which is the peak to peak variation of the distance.
so to see why we can see on the right three signals. black is ekg. the blue and the red are actually two ppg sensors mounted on the wrist band of the same person. first of all, the first thing we realize, look at the time
resolution. the peak of the ekg versus the information you're going to get from this kind of wearable sensors. this has nothing to do with a particular company sensor. this is about the process. one is about polarization of the
heart and how electricity shows in your sensor. the other is blood flow, a slow process, no matter how well they make the actual sensors, et cetera, our ability to do detective detection will be on the order of 20 milliseconds, versus 1 millisecond of ekg.
not all heart sensors are created equally. ekg gives you different projections of the heart on different planes, that's why it's so useful in assessing cardiac held. ppg is not about electricity. we have to keep that into mind
and we also see see these sensors were mounted on the same person at the same time. we see even the two sensors at the same location has variability, which means the data that will come out will have that variability, and we need to deal with that.
count be steps on treadmill, we've seen different sensors, sensors that are apps, on the mobile phone, then wearable sensors, on the right three pedometers, we can see how variable the simple information reveals itself across the platforms.
we can recognize gestures relating to smoking, eating, oral health like how many times do you brush or floss every day? we're talking about how the come a can be intrusive, think about something measuring every hand movement of your daily life. i'll leave that picture with
you. we have to understand that sensors are just a small part of the big picture. the sensor doesn't stop with the raw signals. we need-- people talked about deciding-- using data cleaning, you know, et cetera, but there's
more processes. we need to model the data from the physiological process and link it to the outcome we're trying to measure. this requires both doing subject level models, personalized, and population level models so we understand the physiologic
process we're trying to measure versus the raw data from the one more remark is basically when we talk about behavioral inferences what the sensors are providing is completely different from what we're trying to measure. for example, to get a smoking,
puff, we need to analyze respiration together with the wrist motion to understand when exactly you're smoking. there's no physical smoking puff sensors, a good work to be done in algorithms to link. to summarize, wearable sensors have to account for variability
between individuals, between devices and between mounting devices on people. sensor placement, body composition, geometry affects how the sensors function, and to make a sensor is not about giving you raw data. successful design requires
standardized measures from the highly variable raw data. this requires you to do realtime personalization of the sensors so it can adapt to you so that the outcome of the sensor is always standardized across subjects but the input is different because you are
hopefully we will be able to use the data we collect over time to enable the parameters as well as our interpretation of the data. >> i'd like to point out as dr. campbell is approaching the depiction of captain piccard was the most repeated at the comic con-- comic con convention you
could be on a roll. i hope you had enough caffeine to keep going. if not, there's plenty over >> cookies are coming. >> cookies are coming to. a pleasure to be here, to talk to the community and broader audience.
i was asked to talk about passive measurements of behavior, i'm going to focus on a student cohort, student body, and talk about what are the capabilities of a smartphone by itself in the background to do passive sensing. i'm not going to talk about all
these gee whiz great wearable devices, but i want to talk about what you can specifically do with a smartphone in terms of behavioral inferencing. think back to college days, that might have been yesterday, or quite a few years ago, but college is typically about going
to class, studying, dealing with friends and families, and deadlines, right? in my case, perhaps partying a little bit too much. students, as bonnie was talking about, concerned about greats, professional development, relationships, roommates, and
health issues but also financial, cultural and safety issues on campus, sort of the concerns of students today. college life is complex. so what happens if life throws you a googley? i don't mean one of these guy "googley," not bad if this is
thrown at you with research money but what i mean is cricket, it's like baseball but much more fun. and we hold the bat down here, and not up here. so googley actually is a certain time of ball delivered by a right-hand bowler, it has a
really deceptive bounce that's devastating for the batsman, typically the batsman is out. this is a picture of my brothered quart, a great cricketedder and science guy, life through threw him a googley when he had a depressive episode and dropped out of college,
diagnosed with bipolar disorder and had a difficult life because of the illness. it got me thinking 30 years later, ed was into technology, what would our millennials do today that are so obsessed with screens and technologies, to sensor behaviors not only
predict future behavior and illness or onsets or relapse but intervention, so things you're familiar with today, you can't open the paper, doesn't that sound antiquated, open a paper, and read headlines this this. more college freshmen report having felt depressed.
bonnie talked about the rise of suicide on college campuses, and here it talks about those who feld overwhelmed by school work and other commitments rose 27 to 34%. there seems to be a rise of suicide on campuses, a serious dartmouth college, where i'm
from, hanover, new hampshire, it doesn't look like this anymore. it's beautiful. people are walking around with smiles on their faces but last year statics were 11% of students with depression, 12s% reported impact on academic performance, 28% have seen a
mental health counselor over the last year. as faculty, thinking one in three, that's pretty high. of course what i claim is faculty, and i can only talk about faculty because there's many stakeholders in student health on campus, but most
faculty are unaware of what their students are struggling with beyond grades. we really don't know what's going on. so to get into the herd and find out what's going on i did a small study where n equals 48, terribly small, but i would
argue the dat is very deep. 10-week term, in spring. we put a smartphone application on all students' phones and collected continuous sensing data, pass irv sensing data and ema and some mental health surveys, the cohort was 10 females, 38 males, cs students,
results are probably biased, 30 undergraduates, 23 caucasian, 23 asian, 2 african-americans. let me tell you about the we put this application, student life app on the phones. what behavior could we infer? no fancy devices. you have to excuse me if i slip
into any sort of loose clinical description here. i'm a computer scientist, but what could we infer? well, activity. you can infer sitting, walking, standing, running, that's embedded in the operating system of all smartphones now.
five years ago had to run a classifier, we did but it's in the operating system. we can infer accurately face to face conversations, duration and frequency. we're not talking about listening in on people's telephone calls, using the full
mics on the phone to actually infer conversational data around a person, and one of the early panelists said if your phone is in your pocket you can't use the microphone. that's not true. the microphone in your pocket, the it's powerful.
we're not recording data, just inferring speech is present. surprisingly, without actually ever using an alarm clock, or interacting, your phone knows how long you sleep by using sensors such as accelerometer, light sensor, sight sensor. plus or minus 30 minutes,
without you ever touching it or telling the phone anything. it can do it fairly accurately. we also can infer whether it's-- [ no audio ] whether the student is study or not, and are they focused while they are studying.
here we use a combination of location, sound and activity, to infer this automatically on the similarly we can infer partying. at dartmouth college, all the parties in the greek house, we use location, sound and activity to infer partying. we collected this data over ten
weeks, i made this publicly available, anonymized, it's on people starting using it for research to discover things we never looked at. what sort of behavioral trends could we pick up from this data? well, my colleague over there, it's burned into his brain, this
is the timetable at dartmouth. the phone automatically determines class attendance. it knows location, it can cluster it, monday through friday, over the day, for the whole term. the lighter it is, the cohorts are more likely to be in the--
more students are in those slots. the darker it is, the fewer students are in those slots. the common theme with this study was coming to the students with one class, other classes from 18 departments, this is all their sleep data across ten weeks.
some students went to bed before twelve, most went to bed between twelve and three, and we had a bunch of vampires at the end of the scale. i'm going to show you quick fast slides that all have the ten weeks across the x axis, all have the mid-term period, and
this is the sleep data. what's not shown here actually, well, it's not shown here, there is a word flow that comes along here and drops down because this was a keynote slide which has not played nicely when converted to powerpoint. imagine a work load curve as
students come back from spring break, they don't sleep they have for the first week. they get some sleep sleep drops through mid-term period, they get sleep and it plummets during finals. here is some really-- for me this is interesting.
the top slide, top curve here has a-- i don't know if this is going to work. the top curve here has the duration of conversations, and this curve here has the when they come back from spring break, they have fewer longer conversations which i can only
think of being two minutes, only can imagine being social conversations, as we go into the mid-term period it flips over where they are having more shorter conversations, more business-like, maybe stressed out styles of conversation. it picks up toward the end with
longer conversations, physical activity to the mid-term and then it symptoms. this is ena data. they come back from spring break, top curve is positive affect, feeling good about bottom curve, by the end they are highly stressed and feeling
terrible about themselves. this is class attendance, the curve is not there so i don't embarrass myself. they go to the gym until mid-term and stop going. when do students party and when do students study? this curve shows you that
wednesday night is a big party night. this is automatically inferred from sensors on the phone. no emas, no self reports. so wednesday is the big night to party which used to be friday night when i went to university. friday and saturday are big
nights to study-- to party. over here you notice monday, tuesday and wednesday, those are the biggest nights for studying, before the party week starts. so here is one serious mental health result, correlations between automatic sensing data and phq 9 depression scale.
those who slept less had fewer conversations, around fewer people, more likely to be depressed. what's cool about this, this is automatic sensing data on your phone that is being correlatessed with phq 9 depression scale.
24/7 sensing is here now. it wasn't here a year ago. why is it important? we count significant correlation between passive sensing data and gold standard validated surveys, i'm going to get kicked off now. where is it going in the future? the phone will be able to
predict gpa, we can already do that between .17 of a great point, it will predict depression and mood disorders, build personalized relapse models so we can do intervention. what's going to happen in the future if life throws you a
googley? mobile will detect it and mobile will deflect it. >> a pleasure to be here today and chat about the slightly maligned self report. i'm going to argue-- i'm going to haring-- thank you, jeff.
i'm going to argue today that actually self-report is tremendously important, for this initiative, and it's important that we do it as precisely as we can. the way i thought about this talk was that instead of presenting you some studies from
our center, we do lots of work validating self reports and developing new ways of measuring self report, i thought i would try to give you more of a conceptual overview to make the case that self reports are critical for this initiative and doing it well is critical.
so with that said, self report sounds dry. it's not a real exciting sexy kind of topic, but i want you to take a look at what they are. self reports include patient reported outcomes in health care settings, critical to the medical environment.
medical and social histories for medical trials, opinions about all manners of topics are self report as well as more traditional psychologic status, whether a person's depressed or optimistic, whatever. the realm of self report is incredibly broad.
and that's one message. but why should this pmi effort collect self reports? well, there are internal states that we simply have no other way at this point of getting at, so you see there we have pain, stress, fatigue, anxiety, craving, motivational states, we
don't have sensors for these and as a matter of fact, we need to measure these things with self reports if we want to develop sensors which will try to approximate them. actually i think the last talk showed you that. there was ema in there.
ema is self report. we need that in order to figure out what's going on with our sensor data frankly. i think there's also some unique things that we simply can't capture without going to accept report. observable states, i put a bunch
of symptoms there, asthma attacks, coughing, consumption, drink, food, medication, exercise, lifestyle. now, these are in theory observable things, sure, but no one does that. usually. because it's too hard.
we asked people. the point here is that-- let me finish this and make a point. for environmental characteristics, such as where people are, what they are doing, who they are with, social support, and for observable exposures, then we collect
sociodemographic information from people. we use this data in most scientific trials even if the focus is on genetics or physiology, we use this information, collect this information, and most of the human trials, it's imperative
that we do it as well as we can. in the last point i say self reports are subject to various biases and errors, ability absolutely true. lots of things that we as creative humans have evolved to do in order to make ourselves feel better, to make ourselves
view the past in particular ways, we have limited capacity, so don't ask me how my pain has been over the last month because frankly i can give you some rough aapproximation, and that will have a signal to it but it won't that be good and we need to be precise in what we do with
self reports, that's where a lot of our effort goes. so why-- the way that we try to get at self reports in a more precise way is by going and doing realtime collection of data or near realtime. and what i mean by near realtime, end of day diaries for
example whether we ask for a relatively small reporting period that day, and we collect that at the end of the day. we've done studies validating end of day reports with many momentary reports throughout the i'm not going to go into those. so we need realtime, near
realtime reports in order to increase measurement precision. then we need it to provide much greater insight into people's experiences. this is what's important to how they are feeling. if you ask folks about their pain for the last month, that is
a very gross way of understanding their pain experience or any other symptom experience that they have. we need to be more into people's natural environments, following them as they go through life experiencing what they now, the other thing about
realtime and near realtime is that it allows new associations to be discovered that simply cannot be discovered without high resolution data. so for example we can look very-- ema ecologiccal study in a dense way for many days, sometimes many weeks, and from
these studies we can put together patterns and we can use particular kinds of sampling routines in order to understand people's trait-like feelings and experiences, as well as by targeting certain aspects of their day when certain events have happened so we can
understand things like resolution of an event, or response to a medication. these are already been used in examining gene and environment interactions where we know for example that people with different genetics loadings have differential reactions to
momentarily measured stressors, small things that occur to people throughout the day. how should we do this? this is largely about mobile technology, that's great. we can use mobile technologies to do end of day diaries, i'm going to argue in a minute that
end of day diaries may be especially a good way to go for a long-term intensive study like the pmi initiative or we can go to within day capture ema and we can get very high resolution but the burden is much higher as so there's going to be as we've heard already a balance between
burden and resolution or intensity of data collection and that's something we've played around with a lot and i can tell you more about that offline. and then there are actually ways of doing micro experiments, if you program your devices that are collecting ema data
appropriately and you use the various kinds of random sampling, event-driven sampling and time-driven sampling you can do lots of different things as has been done in smoking research area, for example. there are decisions and challenges in collecting self
reports in this initiative in particular, the platform will matter. i just have to emphasize as others have, we've heard of usability, user friendliness, incredibly important for a long-term endeavor like this. much has to be done on how to
make this work for long periods of time. i won't talk about connectivity but that may matter too depending on reports and feedback you want. i think there are some things to be learned, some lessons to be learned from literature on ema
which now is 20-something years old, and that in thinking about a recommendation for this initiative, it was sort of hard to think about it because i don't know nor does anyone else know exactly the kind of questions that we want to and usually when you're
structuring a study like this you think about the questions and then you back into the that's pretty hard to do. so i think that the committee is going to have to think more about critical questions and design a protocol to get at those because you're not going
to be able to do continuous monitoring for 50 years. i don't think anyway. another challenge but i think it's an exciting challenge and goes into we can learn new things, if you collect dense self reports, you can summarize experiences in ways that may be
much more meaningful to people than what we currently do. asking people about their average pain over the week is one way to learn about their however, there are many others. the pharmaceutical industry has started to take this up and is getting a bit more savvy.
we have a project going trying to ensure what kinds of ways of summarizing dense information are most meaningful to patients and most sensitive to the medications and treatments that we give them, not only that, what kinds of ways of summarizing data are most-- are
most of interest to regulators like fda and ema, or to other folks, for example the insurance agent, not agent but the insurance industry, and that's something we haven't paid enough attention to. finally, in my last 20 seconds, i'm going to advocate that we,
you, consider an intensive burst design for collecting some of this kind of data. this is coming out of the lifespan literature, a burst design is where you know you're going to be tracking people longitudinally over a period of time but you decide that you're
going to do, for example, one week of intensive measurement, or two days of intensive measurement, at pre-set times, and this is being done in the longitudinal-- like i say, the lifespan area. however, i think that there's actually a trickier way, we're
trying to implement at usc, what i call sentinel event monitoring where you do a loose tracking of what's going on in people's lives, say for example on a weekly basis, and you use that to trigger some of your bursts so you're following people at times when interesting things
are happening in their lives, or when they are anticipating interesting things happening, like retirement, like an illness, like a death of a spouse or close person. things that could impact health. so in sum, thanks for your attention, and i really do think
that mobile self report capture, it has a lot of challenges, but i think it would add much to the pmi effort. >> thank you all. as the working group has been struggling with the first order of base, and that is the underlying principles by which
to formulate this cohort of a million people or more, it's been debating about the relative priorities of investing in the acquisition of that cohort and if you will, sort of minimal data set. my question is this. i'm impressed by the previous
panel's observations, the last question and many observations specific to behavioral dynamics echo early cautions about the frames premature nature of data acquisition at large scale basis. so is it-- convince me otherwise, if you will, that we
can set aside in the assembly of the cohort the immediate concern with the acquisition of data by mobile technologies of this nature, that we can think about that as the immediate next several stages of inquiry, as the cohort is assembled, as the questions of the day emerge, or
what are the other options? >> let me impact that question, a great question. can we trust existing research and advances in sensing going forward? is it valid enough to use or is all this sensing just good enough for, you know, consumer
sports sort of applications? my feeling is that i can only really talk about my own space in terms of smartphone sensing, and so i think it was kevin talked about the trust of a fitbit in comparison to a clinical grade accelerometer. i would be interested in asking
him what do you think about the top of the line smartphones in terms of their accelerometer or initial sensors. my feeling is if you take one thing that i feel fairly confident about, five years ago we were like many of those building classifiers doing
machine learning, programming phones, to do activity recognition, which had quite high performance in terms of accuracy. 80, 90% accuracy. along came apple and google and embedded classifiers into their phones, into the operating
systems of the phones, basically now collecting hundreds of millions of accelerometer data and inferencing to evolve their software on their phones to be of high grade. i would claim that those classified doing activity on smartphones are probably rock
solid, great to build something on. similarly, if you can imagine going forward looking at other types of behaviors like i'm particularly interested in a fitbit for the mind or fitbit for the brain, trying to infer psychological states and agree
with arthur you do need em area-- ema or self report. if we could get a sensing google could put in android to infer stress or anxiety or depression is the lifespan of this pmi five or ten years, if it's five i doubt any will be embedded in operating systems but in ten
years i think there will be, and people like ginger here would probably have a voice in the commercial side, not just academic research. >> one concern, coming from the momentary data capture field we're very concerned about using the principle of random sampling
throughout the day to try to understand in an unbiased way what people are doing. to the degree sensors are being used in a self-selected way when people are in pain, not in pain, i start worrying about how valid the data is in characterizing them generally.
maybe for continuous monitoring that may not be an issue but a lot of devices we talk about sample, and we let people sample when they want to sample, that i think can be fraught with dangers, so i think it's same thing with social media same deal, when do people tweet,
when do they do google searches? >> so just picking up the clinical grade versus what is in our phones and wrist bands, there's literally a handful, less than fingers of a hand, companies that makes the actual semi conductors sensors that measures devices, so if the
clinical grades have, you know, some leg up, it's about the calibration, it's about, you know, having that form factor being used repeatedly over time. so obviously, all the wearable sensors we'll have will match the quality in terms of raw data, we're going to catch up
and exceed clinical grade spirometry because of our ability to calibrate. we you will understand that wearable sensing puts limitations, twelve lead, two lead, no-lead ppg sensors, i try to highlight there's valuable information, but there already
shortcomings, it will have limitations. i think it's important to understand those, as an engineer i don't feel show me how it works, i say show me how it fails because that defines how it can be extended to different populations so i think in terms
of ability to capture raw data, we're there. in terms of linking physiology we have some way to go. >> let's open to the general [off mic] >> use your microphone. >> sure. so this question is around the
self-reporting, how do you think about demographics, you talked a lot about utility and ease of when you start to have different levels of education or just the type of knowledge around even normal phones, smartphones, i'm curious how you're thinking about incorporating that.
>> with the question being can people of various demographics actually do this, generally that's been a positive picture, surprisingly older folks do very well with some of the smartphones, in the ema applications, i imagine with the other sensors.
so most demographic groups do okay but i think you're absolutely right in the sense too that collecting data in realtime is not for everyone. it doesn't work so well for surgeons and bus drivers, for example. so that's when you have to start
thinking about who is the population, where am i going to get the most coverage, that's where you can go to an end of day diary, and there are new variations of end of day diaries, something called the day reconstruction method, which is a way of combining time use
information, that's how people spend their time, that's a field that's been around for 40 years, parsing the day into episodes and then finding out once people got that back into memory what's happened during those segments, and right now we came up with the drm about ten years ago, and
there's something like 1400 citations of people using this at this point. i think this could be done, it's being done in large scale nih studies, health and retirement for example, as well as by the w.h.o. and oecd. hello.
so i just wanted to make a brief comment of an issue that's been raised by both panels, measuring diet, we is an issue we have given a lot of thought in nurses health study three, so to put it in a nutshell, what we've decided the best current solution for that is to try to
incorporate into mobile technologies existing versions of dietary assessment tools, because-- and the other thing we have come to realize is that there's two elements of diet that we could separate in using mhealth, one is the composition of diet in which the current
best solution we think is to adapt existing methods into mobile platforms, and the other domain is that diet behavior, when do you eat, why are you eating at specific times and specific places as opposed to others. for that we're actually much
closer to getting a very good passive assessment using ema, build-in phone sensors than to getting to the actual composition of diet, a much more complex question that i don't know when are we going to get hopefully soon but it's definitely not ready for prime
time today and probably not ready prime time in the next three to five years. maybe i'm wrong. one thing we have tried is to go outside our comfort zone trying to find solutions, so we're organizing an event we're calling a hack-a-thon inviting
programmers and hackers to around the country to address the questions we haven't been able to solve in to address precisely these questions regarding diet behavior. now, the other point that i wanted to comment on briefly is the concept that has been
touched by both panels regarding that not necessarily everybody in the cohort this size is going to be using sensors but everybody is going to be using-- either answering questionnaires or having an app or something but the sensors, who gets what sensors at what
time may vary during the cohort at any one time. we don't think this is necessarily a disadvantage, and actually one potential application we've seen for this type of different ways of collecting data is to create measurement error correction
models, so you may end up getting much higher quality data with sensors in a subgroup of the population, but that doesn't necessarily mean that all is lost with the rest of your study population as you can use the data that's collected across the cohort to validate and create
collections for the larger groups of people. >> one comment since i'm sitting here. i really think, it's not my field so don't get discouraged if you're working in this field, i think people reporting what they are eating repetitively by
entering data is not going to work. and i think-- because i just think it's tedious and unless they are really interested in their diet, the general population of a cohort this big wouldn't be interested i think unless you can really automate
i think the future of behavioral sensing is all about the passive nature, zero cost to the user. so i think what we can do here though, there is light at the end of the tunnel, i think clearly computer vision technology is the way to go about collecting dietary
information over the long term, and computer vision isn't there right now to be able to do that in a robust reliable way. people are working on it, maybe google really worked out, they could solve it just because of the massive amounts of data they could collect but i think the
future is an automated way using cameras and computer vision in solving that problem and not people entering information. richard? >> to what extent is it known how consistent the interpretation of various kinds of measurements can across
populations? does a pattern of measurements that reflects stress in one group of people mean stress in another group of people, or partying or a variety of things? i wonder if the pmi will adopt these does it need to be attentive to other
characteristics of individuals being monitored? >> you said stress has about 47 meanings. if you're talking about self-reported stress, you know, the area of psychometrics has developed to fairly good science, and this is one of the
main questions. do different groups of demographically defined folks-- do they answer items in the same way? there's differential item function which is do these items respond to groups as i said in exactly the same ways, but there
are other issues. we currently are under an nia grant trying to figure out whether people self report about their pain and their fatigue, their well-being and their general global health in the same way when they are older, younger, or middle aged, just to
make groups out of it, because we think that people may use different ways of comparing themselves to the world at different ages, so these are questions that are being examined, how it works with the sensor data, i don't know. but it's a great question and
it's something that needs to be resolved. >> how does it work with the sensor data? >> so maybe i can just talk. so as i was trying to allude to, how to do the stress measurement, how the body's nervous system activates
regarding negative effect associated with stress. so that part is going to work across the population, but how that activation is actually perceived into stress, that's a personalized process, you know, the fact that i might be a person that not perceive
activation forms into stress, so that personalized difference can only be, i feel, resolved through simultaneous emas, et cetera, to see how physiology goes to the perception of stress. it's a complex question. >> this takes us back to kevin
patrick's earlier observation about the importance of the movement from disease, illness to predicament, and the significance attached to individual experiences, very enormously, but these kinds of factors and how we can get our arms around that in some
comparable ways, the challenge you're articulating. >> please. >> so, yeah, back to the mental health thing which i think is particularly the dartmouth-- a great opportunity for getting in early and doing something, even if you can't pinpoint by android
or whatever exactly and predict, can you just put a sort of threshold on sleeping behavior, say, and people and say, oh, your sleep is dipping down, just like a mother, you need more sleep, you need to go to the gym more or whatever, can we do that sort of thing and do it sooner,
and then the second point is i guess (inaudible) which i'll throw out, the act of changing what you're looking at, we heard people want n of 1s, they want to see changes, but as researchers how are we going to actually stop things changing long enough so that we can
compare to all the people out there who aren't in the study. >> so here's a straw mat. i think first of all if you're interested in what phone to use in this cohort it shouldn't be an iphone. there's a strong word for you. because not that i'm sponsored
by google but because the-- (inaudible). >> i actually am sponsored by here is the thing. it's not a continuous sensing it won't allow you to run continuous sensing passively on an iphone, with the research kit it seems opening up a bit
there but i'd say android is a great environment for doing sort of continuous sensing, passive sensing, maybe the i phone will there's a phone from microsoft as well but we won't talk about in terms of the question about what's feasible in terms of getting to psychological states,
whether it's physical stress or psychological stress or, you know, mood disorders, what we did in our time needle of study, and i can't claim with just 48-- where n equals 48, computer science students at an ivy league university would general rise across the country
to different types of colleges but we found significant correlation with passive sensing we don't fully understand why that's the case but we found it for perceived stress scale, for phq 9, flourishing and loneliness against the gold standard surveys.
if you could repeat that different universities and other people could do that and find similar results, it would built up a confidence that passive sensing maybes from smartphones could be used for physical health but also psychological the next step beyond that,
probably beyond the cohort, take the correlations and take the understanding of those signals and build predictive models to predict something like the questioner just said, to come up with something fairly simple, reasonably challenging, to predict something like a phq 9
depression scale from the sensing data. beyond that you're getting into the domain of how do you do personalization for n equals 1 which i don't think we fully understand. i see a road map to getting to a point where one day we might
have a validated psychological sensor, probably running on an android phone. >> arthur, you have the last word. >> the person is gone, acting about reactive arrangements, is the act going to change what's going on, because ema has been
around for quite some time, people asked that question, we're hitting you 10, 12 times a day does that change how you report? there's in four or five studies very little evidence that things change over time, at least over a several-week period.
that's surprising, but that's what the data is saying. we design one study to look at that in particular and saw no change in level or variability or relationships among variables that were being measured they moment, the best we could figure out to do to get at this
reactive issue. >> please join me in thanking all panelists this afternoon. >> we'll take a 15-minute break, we'll have dr. collins announce it's reassembly, a quarter after >> the first speaker is gary bennett, dr. bennett is the beneficiary bishop mckerr mom
family practitioner of psychology duke university, and dr. donna spruijt-metz, at the school of medicine at usc, and dr. david gustafson, director of the university of wisconsin. the demographic of the united states are changing dramatically.
as of today currently 40% of the u.s. population identifies as non-european in origin, the vast majority of children are not european background. these are real implications because when you look at common diseases like asthma, and my slides are not working.
there you go. those are the demographics of the united states, very important because when have you common diseases like asthma, when you see the prevalence, it varies dramatically by race and it's easy to argue that this is a sampling bias but hard to
argue when you look at death rates. the federal government recognized this and created some initiatives. in 1980 we started the women's health initiative. 1993 congress required the revitalization act to focus on
inclusion of ethnically diverse populations as well as women in minority populations. one of the questions we've asked, how have we done? in the ten years since the completion of the human genome project more genes have been identified in the last ten years
than probably all the history of medicine combined. but we've made some shortfalls. here is an example. this is a publication we published in 2011, just looking at how many non-european populations have been included in modern genetic studies.
this is a travesty, the data speaks for itself. we want to make sure we don't repeat mistakes of the past. everyone knows the idea of food deserts, everyone knows that well and i don't need to speak to that but what we've created are scientific deserts, we don't
understand the biology of how disease operates in different populations so we have scientific deserts plus health deserts, it leads to health disparities, what's what we're trying to prevent here today. this is really important because when we have data like these,
this is a package insert for one of the most commonly used asthma medications in the world, advair, this is problematic, and important for us to understand the biology of how drugs operate in different populations. i'm going to turn it other to the next speaker because we need
to address how do we get pmi out to the community, how do we address everybody and make sure that our technologies are not only applicable to people in silicon valley but also appalachian. i'll have dr. gary bennett speak.
good afternoon. it's my great pleasure to be here today. this is a fun conversation for i've been coding and tinkering for most of my life and it's excite be to be with so many in other words. i run a research center at duke
where we're very interested in leveraging excitement around new mobile devices and sensors and the like. we're particularly interested in application in places like this. and like this. we're predominantly interested at duke digital health trying to
understand ways to utilize mobile health technologies to try to remediate health disparities. i've been doing this since the early days of 2000 and 2001 when if you proposed this kind of thing reviewers would say, yeah, but poor people don't use
computers. that was a real comment we heard pretty frequently. we've come a long way. it's extremely exciting to talk about really going after a million people, allowing mobile devices to be a major part of that effort.
in looking at videos from talks at previous workshops and agenda today, i grabbed questions that i think are probably of most how can we engage diverse groups, is that possible? what bias might we introduce and how might we mitigate through use of technology sampling and
i'll talk at a level upstream from the conversations this afternoon, talking about data acquisition, getting the data i would love to talk about challenges of sampling but we're going to talk about how you get the data into the datasets. the first, to do level setting,
medically vulnerable populations, racial ethnic minorities, socioeconomically disadvantaged and live in rural settings are connect and disconnected at the same time. they present a host of challenges when you're trying to do this work.
we've heard about the 64% of the population who own smartphone, that masks a lot of variability in that metric. it's the case that more blacks and hispanics own smartphones than do whites in the united states, but if you look at this by income, the situation is a
little bit different. before i get there let me say one of the things that's most interesting, if you remember the digital divide from the early 2000s, you thought about racial disparities in connections to broadband pipes and access to computing
today we have new divides that are i think mostly a function of the fact that racial and ethnic minorities are more likely to use technology, using advanced digital data features of mobile phones. if you look at this list of activities, all of these are
disproportionately more likely for blacks and hispanics relatives to whites. if you look at social media sites, these are disproportionately black and latino, in some cases. this provides and untapped way to reach populations we've been
calling hard to reach. if you look at income and education we see a different pattern. somewhat less likelihood to own smartphones for those at the lowest ends of the socioeconomic spectrum, this is going to complicate our efforts to roll
technologies out to the million. i want to highly the group of 19%, smartphone dependent, who utilize but don't have access to other times of broadband pipes in their lives. they are disproportional lit racially and ethnic minority, tend to be low income and have
low education. this would seem like a major opportunity, to boost the likelihood of being able to use mobile phones in a large their access is quite tenuous. if you do work in the area you realize cancellations of service, degree to which folks
have exceeded data caps makes this a challenge to roll out in a pragmatic way, it's a challenge and opportunity. the 10 to 20% of the population at large who are disconnected depending how you look at the numbers. disconnection is more prevalent
among blacks, hispanics and those in lower education and income groups. there's a host of challenges but opportunities as well. our question is how do we engage the medically vulnerable with mobile health. i'm bullish on this as a
possibility. let me walk you through the principles that emerged from our work and others. the first point i want to make, a lot of discussion about smartphones, i think one of the things we realize early on in this work is we're still at a
point where smartphone penetration has not reached the level that makes wide spread dissemination of interventions in my case really feasible and able to reach into the medically vulnerable. it is the case that 91% of us have a mobile phone.
most frequent activities, feature phones or small phones, are texting and voice calls. we can smarten texting and voice calls to do a lot of self report assessments that art was talking about a moment ago. that's where i think we've hit our sweet spot in using
technology to reach the medically vulnerable. about this issue of precision and how we might reduce measurement complexity so i won't spend time on this but will illustrate quickly this is a key point in reaching the there's a temptation if you're
like me to try to ask people to hook as many sensors on as youk particularly in the context of intervention where we can use those data in providing high quality treatment. the problem is, this is a real example, this is a nutrition label, also a survey, that's
widely used in primary care, we used this in our studies and asked questions like this. take a look at this nutrition label and tell me if you ate the entire container of ice cream how many calories would you consume? the answer is 1000.
most of you got that. i'll tell you when we give this in our sample, i work almost exclusively in a primary care setting, most often community health centers, just about everybody is in poverty, work settings and urban settings, the most frequent response is no
people will stare. they will push it back. they will look at it. push it around the table. we've had several people cry. we like math. i like numbers. real people don't. i think one of the challenges
that we've encountered is you have to be very careful when asking even things that seem very simple in populations that have had difficulty in education system. what we do is we've designed a somewhat different approach and give people-- have people sit
down at a computer terminal, fill out a short survey, we run the data. i'll breeze this so forgive me. we take responses, throw it into the cloud, run it against algorithms that go into a large library of behavioral goals and we spit out four goals we want
people to focus on. the name of the game in obesity treatment which is where i do most of my work is in long-term you have to figure out a way to develop intervention people will use for long enough to derive clinical benefit. the best benefit when people are
steadily engaged in self monitoring over a long period of time and give common sense easy to understand goals and ask them to self monitor them, in contrast to asking them to monitor their calories. we give them technologies we've developed, wide range of
technologies, and ask them to go off and self monitor. an example of of a text messaging monitor here, instead of asking you to count calories or last your food, how many sugary drinks did you have yesterday and they can response simply with text messaging and
we can provide feedback. we did this in a group of vulnerable women who were obese to try to prevent weight gain and asked them to monitor weekly using interactive voice response calls. it's an automated telephone call, thousands of hours of
audio voiced by a human actor, piecing together the call and the feedback so everyone feels like they are hearing something it takes 3 to 5 minutes. at the end of them self-monitoring they get tailored feedback. you get 72% adherence in an
impoverished group of predominantly african-american rural women, 72% adherence, for me it's a major win. they take the calls and go through the self monitoring episode. you can imagine the things fitting the things art was
talking about, the end of day diary, a similar thing. you can do that with a medically vulnerable population. one study, people chose whether to receive the automated telephone call or go on the web to do self monitoring. you see it may or may not be
surprising, the telephonic technology works better than the web. people tracked for longer, a two-year trial, tracking weekly over two years, high poverty group, homelessness, people using food pantries, urban, african-american, high rates of
poverty, primary care based. you see they use the interactive voice response calls in much higher rate than the web but importantly when they use the automated telephone calls they are less likely to stop tracking over the course of that two-year period.
we allowed people to swim back and forth between the voice response and the web. when people have that ability, the people who avail themselves of opportunity to switch are those least likely to stop using. the message here is give people
options, and not to just presume that everyone will use an app. and we recently have shown if you combine this interactive voice response with text messaging you get 80% adherence, medically vulnerable primary care population. and this idea is not a new one.
this is what facebook does. facebook doesn't care where you access through text messaging, mobile app, web app or the web, multi-delivery channels is a true approach to maintain engagement and works in the we want to consider user experience to minimize bias.
there's talk about sensors. we learned you can't use a sensor that connects to someone's ankle and expect that to work because it may heighten concerns about criminal we've also done fun things through e-mail but it's hard to use e-mail in populations that
are manual labor occupations or work in the service economy. we use a scale like this instead of attaching it, we asked people to weigh themselves every day, instead of attaching to blue tooth or wi-fi that collectings to a cell network like a kindle, put it in rural homes in north
carolina, mean average 60, we ask for daily, they weigh four days, a good indication but what's important here is we've minimized barriers to them being able to use this, haven't presumed they have broadband access and the like. in the interest of time i'm
going to stop there and say that the final piece is one about user testing. a lot of what we do is user-centric, one of the most important things we've learned, as steve jobs, a lot of times people don't know what they want until you show it to them.
a lot of times people don't know what-- they may not know what they will do and so it's critical to continue testing we can test to higher engagement and we've shown you can do this in medically vulnerable populations, i'll leave it it's an honor to be here.
i have ten minutes to tell you about why children should be part of this. so here i go. first-- oops. let me just dispel a myth and that is that children don't have this is new data coming out of the pew study.
age 13 to 17, 88% have access to cell phone, smartphone access higher in african-american populations. what about younger children? an interesting study that came out in 2012, i haven't been able to find much newer data except some we have from los angeles
inner city schools. what you see here is about 11 years old seems to be the sweet spot that parents are buying phones for their children. this is really important to kids are-- kids do have phones. why is that important? because we've talked about the
mobile phone a number of times. i want to remind you these are data hungry ubiquitous computers we're carrying around, integrated with wearables and deployables, what our earlier speaker called awarables. a lot of data can be gotten and given back through these phones.
what i wanted to do today is i've worked in children and mobile technology since the early 2000s. i want to share lessons learned about what it's like to learned with particularly minority children, all my work is in minority children.
one thing i know is that context is king. these mobile technologies give you the ability to understand behavior and health in context. in the settings where these things are happening. this was a case study knowme networks that interfaced with a
mobile phone, realtime data analysis and feedback to the children. we designed this with the kids and for the kids, user-centered design from beginning to end. they were involved the entire we did decrease weekend sedentary time by 170 minutes,
profoundly sedentary kids. here is some lessons i learned working with these kids. if you want them to trust you, you need to be accurate with the feedback you give them. so what we did was interfere in sedentary time, and what we did was if they had been sedentary
for a certain amount of time we said aha, we sedentary, your algorithms need to be accurate with kids because they are so tech savvy. we learned for the kids i worked with if there wasn't a human fairly close in the loop, our technology wasn't ready to to
receive the messages we got back from them. we had to keep a good watch. they like to hear from us. another thing that we learned was we had the surprise users, surprise people who wanted to be part of the research. after a couple of these
user-centered design sessions when parents were in another room, getting some education, we took care of them well, we fed them, they suddenly stormed the barricades and wanted to be part of the knowme study and wear the it would have been a $20,000 programming job because they
wanted to see something different on their watch faces and their phones, but what's interesting to me is that by having children in the study, it was value added to the parents. i can only imagine now with the new cohort we're going to develop that including children
will make the entire endeavor much more valuable to the parents. another lesson we learned, one of my favorite articles, omg, don't say lol, they know who you are. they know we're researchers. we're not children, we shouldn't
use their language. they are smart and savvy, and really don't do it. lessons from digital natives, i asked a bunch ever kids, are you willing to share your digital traces, phone use, location, are you willing to share that with a researcher like me?
the answer was sure, yeah. with some maybe i would like to opt out of a few things but for the most part, yeah. then i asked them, are you willing to share your data with friends or family, just your physical activity data. um, no.
no. sharings is a different animal with these kids. this is the generation of snap chat, where their data and disappears in no time. they know what they want to share and what they don't want to share.
they are since active to things we aren't expecting, not sense actives to things we get our knickers in a twist about. working with kids is a whole new thing but if we want to build this for sustainability, building for children is really they were born digital.
their privacy and security concerns will be different than ours but designing something like this we can learn really a lot from working with kids. they loved attention from us. they sent us messages, about 14 a day, sometimes more. they said it's like having a
doctor in your pocket just having us around. the born-digital kids expect digital as part of their experience and will be a real asset to a cohort like this. i want to talk about something else, this is my second case study, imagine health.
what we've done here, we've developed mhealth solutions for collecting sensitive data in the field in youth. for instance, if you want to study stress and get salivary cortisol in the field, how many of you have done that? it's hard.
joan, where are you? one of the important things is timing, when they wake they have to do it right away. we developed this app that gives them alarm, we give them labeled vials, ask them to take the cortisol when they get up and three times during the day.
it's time stamped, they take a picture and they send it, we ask stress questions to see how they are feeling. two important things we know exactly what time they have taken their cortisol, which is extremely important, and another thing we have on the back end is
when we ask how are you feeling, are you feeling stressed, anxious, we had a little algorithm develop that if they were feeling stressed or anxious we didn't ignore that. we said tell us what you're feeling stressed about and we monitored that on the back end.
it's these are great solutions for sensitive data in the field. kids are willing and able to do last case study is virtual sprouts, this is a mobile gaming-- it's a game to learn gardening and cooking and healthy diets for kids from third to fifth grade
predominantly hispanic, a few things i would like to tell you about this. interactive gaming is incredibly engaging, it adapts to the child as they learn. i'm strived i didn't hear game-ification today. you need to have somebody who
knows how to do it, it has to change a lot. but the one thing is it's sticky and how we need to make the technologies sticky and make them engaging for a long period and another wonderful thing about these, what you can think of is an interface that's like a
game interface. marybel talked about that earlier, collecting data about how children learn, what they do and don't understand about the data you're giving them. it can collect data on the back end that you don't know you're getting which is really
fantastic. this is also being done in the elderly, by the way. and has been extremely successful. so in closing, i don't want you to forget the internet of we've heard a lot of things about that today.
these are the awarable sensors, they are combining with the wearables so i think the need for wearable sensors will decline a bit as the phones get smarter and sensors get more deployable. i think learning about diet, for instance, in the home and having
the home as a hub is the next generation. i'm going to be involved in that work if anybody would be happy to hear about this after this, i would be happy to talk about it. these children are the grownups of tomorrow. i think that involving them in
this space is the most important step that we can take. >> the next speaker is david gustafson. we're running behind, he will keep it to 12 minutes. >> sure will. >> late november on a tuesday morning, waking up at 7:00a.m.,
making coffee. before they opens the paper she sits in front of the touch screen display and checks her screen, she has an in home nurse visit at 4:00p.m. and notices the reminder to make sure and vote. she's got two new messages, one
sent by her granddaughter katie, more pics of jarrod, and one from her exercise team, there's a new goal set, four 15-minute dumbell sessions for the week. she has yet to let the team down though the exercises can be tedious. she looks at pictures of jarrod
right away. after the paper it's laundry did i, she's got cookie baking on the mind. the kitchen smells like ginger snaps. she ignores alert sounds because she's on the phone with ron. she taps okay on the screen.
you've been spending more time in bed. are you feeling dizzy? you're almost out of warfarin? she answers no. she enjoys picking it up herself, a chance to get air and see pharmacist chad who makes her laugh and so handsome.
she heads to the pharmacy, inside chad pulls up a report and she's been experiencing more fatigue and swelling of the ankles than usual in the last couple weeks. when he asks if she's been taking her lasix pills, she comes clean with him.
reading the paper every morning she developed strong feelings about the election, wants to stop to vote at the community center on the way home but is not familiar with the route. not feeling comfortable as she used to behind the wheel. she says community center, into
the gps and is directed on a route for her preferences, avoiding left turns on busy streets, where a lot of folks end up in trouble. when she is done voting she presses "home" on the screen and follows the route back. it used to be difficult to keep
track of who was coming and it was difficult waiting. an alert sounds, with a notice gwen, the nurse, will be 15 minutes late. christine checks in by inserting a thumb drive into the computer. christine is the only one gwen know who is can stick in an iv
needle without causing pain, why she gets a cookie on her way out and when the survey pops up asking gwen how christine did, a five star rating. gwen found christine using the service depend built feature, schedules appointments because she was rated highly by others
who needed similar services. as gwen is watching murder they she wrote an ipad vibrates, it's a weather alert. it's going to snow tonight and tomorrow morning. she watches a brief instructional video on how to use yak tracks and walk safely
on ice, leery of all things slippery since her fall in the tub so she listens closely. she already knows the knucklehead johnny down the street will be over to shovel in the morning as he always is. a while back she was asked what she likes to do, cooking and
murder she wrote. a community organizer asked if she would be interested in swapping cookies and history chats, both of interest to the young knuckleheaded johnny, for snow shoveling. they struck a deal. johnny would shovel, gwen would
bake and teach. johnny's parents were thrilled to have him out of the house now and again. his mother refers to him as a boy with... way too much energy. gwen is able to keep herself occupied during the day but at night when there's not much
going on she thinks of donald and sometimes a terrible sadness washes over her. gail lost her husband too in the same year as gwen. gwen and gail used to trade off visiting each other in their homes in the afternoons. it made both feel better to talk
even if it was about nothing in particular. when gwen tried e-chess she learned to use the simple video chat and e-mail feature to talk to her granddaughter after she moved a year ago. really wanted to be able to keep in touch.
she learned fast. she hadn't expected to use it for anything but that. eventually she followed the invitations made and joined a chat room, supposed to be a place to talk about cooking but often the talk ended up being about nothing in particular
which was just fine with gwen. she did appreciate when others shared pictures much their cookies, turkeys, cherry pies. it isn't as good as visiting with gail but it's always there for her and somehow makes the sad evenings less sad. saving the favorite part for
lests she writes to granddaughter katie. she pulls up a picture of ron when he was little and sends it for comparison. see, same cooked smile, she writes. clicks on send. time for bed.
>> well, that's the system that we have going at the university of wisconsin in three counties, milwaukee, a suburb and a rural one in richland. it's a system that makes me wish that pmi was phi. not precision medicine, but precision health.
the reason i feel that way, maybe it's obvious there, that the medicine is really important, don't get me down. i had a hard transplant 6 1/2 years ago, and so i like but it's only part of an elderly person's life. and if we're really going to
make a difference in their life, i think we need to go beyond a medical focus. this is when we first got started, we did a process called asset based community development where we went into the three counties and spent time with about a hundred people
per county, and we found that the elderly people said what i want to know is how do i deal with loneliness and isolation? they wanted to know how could they get meaning out of life, and they wanted to know how do i get to a community event? i can get transportation to the
doctor's office, i can't get it to a concert. doctors, on the other hand, appropriately so, said we need to deal with things like falls and depression and dementia and adherence to medication. neither one is wrong, they are just two different views, but
both important views. what we've done in this particular project is to assume that if we deal with the left-hand side, a lot of middle and right-hand side is going to take care of itself. so we focused on loneliness and isolation and things of that
particular type. one of the questions we're supposed to answer is the question related to adaptation. and in that particular area, what kind of adaptations will we need to make? one of the things that's so important, if anybody feels
like, well, the 55-year-olds of today will be elderly at some point in time, so they will be able to use mobile technology. well, pro graph is an anti-rejection drug, it keeps me alive, but it makes my hand quiver. it also gives me cataracts.
and for many other reasons, we change as we get older. to expect we're going to be able to use mobile technology in the same format than everybody else does is wrong. i'm generalizing, and there will be some people who can and others who can't, so forgive me
for that generalization. another thing that's really important to this audience is the importance of keeping things very simple through as much as possible automated data collection. we get open rebellion by our people on the system when we ask
them to answer what to do about a particular kind of question that we've decided on. and so i think it's really important that we minimize the amount of stuff we expect these people to give us. i'm going to pass over this particular thing.
another thing i worry about in the simplicity side is really don't give us too much information, and also for the same thing for doctors. what we do is if a person hasn't had a bowel movement in the last three days that's when we contact the doctor.
not what their bowel movement is every day. they pick things, they say, don't tell me anything unless x, y and z happens and then we communicate to them. the idea of big data is okay but we've got to narrow and whittle it down, that makes a big
finally, as far as heterogeneity is concerned, i think there are a number of different things but i'll talk about the bottom three things there. i worry about elderly in communities there are in internet deserts. so many of the elderly people in
rural wisconsin don't have access to the internet. but yet if we're going to have a million-person study we've got to reach those people. the second thing that bothers me is i never really liked the term personalized or patient oriented or patient-centered things
because in reality, when i got out of the heart transplant thing, it was my wife who kept me alive while i was there, to some extent. she was the one that drove me. she was the one that picked me up. she was the one that cooked, et
cetera. so there's very rarely a patient disease, there's almost always a family disease. it's important i think that we move away from the idea of an individual, the single person standing there in the picture. let's replace it with a family.
finally, high quality data is really important. garbage in, garbage out. we've got to worry the stuff we put in is really good stuff. >> beautiful. awesome. we're on time. i'd like the rest of the half to
be open to the audience. we'd like to solicit audience participation. if there are questions, feel free to use the mics on the periphery. we have some questions for the panelists. these are place holders.
i will challenge them to address them but then again let's open it up to the rest. we had a wonderful spectrum of disease from children to elderly, everything in between. the key thing that i heard echoed was we need to reach out and we need to be nimble, so the
same things we do for someone in a nursing home or elderly home might be different than what we do for someone who is of the low ics community. i want to get input on how do we do it, the practicalities. one of the questions, i'll open it up to dr. bennett, who do we
make sure the modern advances are not exacerbating health disparities or not exacerbating the digital divide? you had a young low ses group, what was your experience, on the nuts and bolts of what to do. >> i think the first one is not to presume that people who are
circumstances will not use i think we have to recognize-- i'm sometimes surprised at how surprised people are that folks communities are disproportionately likely to own smart phones and use them at high rates. if you want to design a social
media app go ask a single black woman who is in poverty because she's disproportionately likely to use facebook and instagram to connect with her friends. i think the first thing is to understand people are using but the second issue is to really approach the design of
these applications with the same type of rigor we would approach any other aspect of our science. that is we may start with focus groups to try to understand likes and preferences and those kinds of things but then to rigorously test and iterate through the process of
applications to ensure they are usable and not allow our own interests and leveraging new technology or the hot new shiny thing to cloud that. i think this point at the end making sure we keep these things simple and that we're designing this in an optally and
user-centric way is the way to do it. >> i just want to second what gary said, i don't think it's a good basic assumption to think that a disadvantaged populations or minority populations technically not savvy. they are often more savvy.
>> to donna, i was intrigued with the case study you gave with the-- think it was hispanic teens with weight reduction, and your comment, be accurate and need to have a person interacting. as we look forward, is that-- i would like to know is that
generalizable? it's like we're sending these things out and stuff is coming in, but how much person to person, either hand-holding or coaching, is necessary, from all of you, actually. >> it's such a great question. i wish i knew the answer.
we've got a couple grants out to try and figure out where it is but it's a moving needle. when i did the study in 2008, 2009 natural language processing was not there to help us. and i know how vulnerable children are. i wanted to know what they are
thinking. i don't just want the data. what good is that without knowing what they are thinking and feeling. we went for the human loop and they appreciated that. now new technologies are making more things possible, so for
instance one of my colleagues i'm working with now at the institute for create irv technologies, bill and skip, have developed things for the department of defense that one of them was sim sensei, it's so reactive to facial expressions, it's in the computer now and
we're using that more on the dave has used more avatars on the phone. we're getting more advanced, and we can start to think about having users slowly pressed out of the loop and almost entirely pressed out of the loop, never always, never always, but we
need to-- every step of the way, make sure we're taking care of needs of our populations, not only because we should take care of the needs of the populations but because we'll make better scientists and better people if we do. does that help?
>> i think that one of the things-- one area where you'll always need some human being involvement is the communication, we do a lot with computer-based discussion groups, and i work with teenage asthma kids. we had this boy called genital
boy. we don't need to go any further with that but you can guess what he was doing on the system. in the case of elderly people, one of the things that we found was that some elderly people tend to to lose their filters a bit, and when they do they are
the most argumentative, the most snotty people. we've done addicts, cancer patients, we've done asthma the elderly people are just plain snotty. and as a result, especially on religion, and on politics. i mean, they are at each other's
throats all the time. so we've had to literally do some calming down and some phone calls and saying you really shouldn't get yourself into that and all that kind of stuff. i think that there's always going to be need for human monitor, anytime there's a
communication system involving >> it's clearly all that murder she wrote. so i think that at least in the intervention literature with respect do the kind of major health behaviors, much of what we know about efficacy of mobile health derives from studies
which humans are involved in one way or another often in the delivery of care. that makes sense. if we're thinking about optimal dissemination of these kinds of technologies we should be thinking about how to develop technologies that fit in the
existing care spectrum but it does make challenging i think interpreting how technologies will exist on their own. i don't think we know that as much as we should. >> i'd like to encourage the audience to ask questions and make use of the microphones.
>> great discussion. my question is one about how much we want to include the global community in this dialogue. the reason i say that is the examples, gary, that you gave will-- fascinating statistics, at the same time when you go to
india or south africa, the kingdom of lisuthu is a great example, they don't talk about higher and lower access, it's low, period. yet they have come up with the most innovative simple, you know, sms simple-based stuff that gets them to the outcome
that's desired, better health outcomes and peace of mind and i can take control and all that. my question really to the governing body is there are plenty of examples, so many of them, where a lot of these disparities, people have figured out very intelligent mechanisms
to solve disparities. how do we bring the international community into this? they solved a lot of problems. we're beating our heads against for. >> can i speak to that? i think it's a fascinating
comment, i disagree. i do think we should bring in the international community at some point, very important, they have done very important work, but if we're talking about place-based work, it is so different in many ways than what we can do and need to do here.
so i think we can learn from one another, but i think that after sitting on many, many panels for mobile health research, that's done outside of the united states, and certainly in india, these are-- it's very different users and very different ways to come downstairs together.
i think we can learn from one another but i think that we will fight with vastly different solutions, my two cents. >> i'm glad that you spoke because i had a question for you actually brought up a very salient point that i think might have been glossed over here in
response to what i learned from my daughter who is here, a high school student, when she went to g.e., they said the theory didn't work because they didn't have women in the room. when i asked you how do we target particular populations, and you used the example of the
southeast asian men who have a unique form of metabolic syndrome, integrating social factors, it's not realistic for to us have everybody in the room so how do we actually do things like that? for example, in the african-american community, the
barbershops are very important, but do we need to have someone from that community inside when we're actually planning the this i'm being very practical and pragmatic. how do we roll this out to people here, and we're at intel but i can tell you third world
conditions are five miles away. i grew up around here and i know we have the international community here. how do we make them feel included? they are funding the dollars that are being used to pay for how do we do it?
you're the one that provided the input so i would like to challenge you. >> i'll give you an answer. in all fairness to the panel why don't you address it first and i'll see if there's anything i should add? >> we should stop talking
because we talked quite a bit. >> you talk. >> so let me use apollo hospitals, largest system in the world, right? heart disease is the number one problem in india. it's fueled lie a number of things, a rampant prevalence in
diabetes and hyperlipidemia and what not. they started a problem, a billion hearts beating, the idea with billion hearts beating was simple, encourage people to take stems, that crosses all socioeconomic strata, something you can accomplish in a
realistic sense, for the most sophisticated person and for the least sophisticated person. it was literally take your route to work, and understand your subway route or bus route, and then you would get an sms, three symptoms, four stops, walk the last number of stops and they
are starting to reap the benefits. now, there are many case studies perhaps the answer is inclusion doesn't mean let's go in and try to find the barbers and put them in a room, that's expensive and time consuming. yet there's so many good case
studies, so the horizontal application is not just for it's for a number of things, not the least of which is obesity and how we fight childhood obesity, getting people more active. maybe an answer, not the answer but an answer, take these case
studies, there are lots of them, well publicized, what can we extract from them? they are foolish if they cut and paste american solutions in their environment. we would be equally foolish if we took they are and cut and pasted them in our environment
but what can we learn? let's take the case studies, they should be part of our research and dialogue. >> now, that i totally agree for what it's worth. >> one thought about involvement, it strikes me as important here, when we first
got into doing work in addiction, i got myself admitted for heroin addiction and state overnight in a detox facility and in a residential facility and toured addiction treatment agencies to see what is going all of our computer programmers have to volunteer four hours a
week in a senior center because i don't want them to have a job, i want them to have a calling. it seems to me that while we can go out and try to learn from each other as much as possible, it's important that the staff that are engaged in putting the solutions together personally
experienced what it's like to be the customer that they are trying to serve. this has been borne out time and again, the predictor of success is how you understand your customer. >> that was a wonderful suggestion, the one i'm going to
incorporate. one of the questions that we want to address is how do we-- i'm asking everybody now. how do we address mistrust amongst community members so people that are designing the research don't necessarily-- may but most likely don't
reflect the people being included or asked to be included in the research, how do we address that? this is open to the audience. >> i'll anxious that question and the previous question too. because i think that our experience with (inaudible)
helps. trying to go through an hmo-based population as opposed to an organic enrollment of peer to peer enrollment, it's very clear that the strategies that wins today for enrolling a cohort today is establishing partnerships with people who
already have existing relationships with the people you want to reach. you just at least we haven't been able to successfully reach people if we do not have an existing relationship, building a relationship saying, hey, i'm a harvard researcher, i want
your data, is not a good introduction card but if somebody says we know about this study, we think you would like it because of a, b andc and my similar experience with this type much study is this, i think you would like it, that's a much better solution to try to gain
somebody's trust than to simply wave the "i am the researcher" card. i wanted to come about your question about inclusion and getting people from a wide variety of backgrounds involved. one very practical things is we want to make a priority to
design for android-based devices, because there's a very strong socioeconomic background in who gets android versus who gets ios device. if you have more money you're going to buy an iphone. if you have less money you're going to get an android device,
even among health professionals. we still see that trend within health professionals. i think a practical thing to do is if you want to have a greater inclusion of people across the country, go for android devices initially at least. >> in your patient population,
it's a very skewed population, can you argue the generalizability to the united states? to your population. >> so, yes, i can. one thing we have done is expanded the definition of who is a nurse as far as the study
is concerned. in nurses one and nurses two you had to be a registered nurse to be part of the study. and right we've now we've included licensed practical nurses and licensed vocational nurses with a smaller salary than registered nurses and
expanded socioeconomic background and racial and ethnic compositions. it's still a cohort of professional women but even among professional women with this type of background we see the socioeconomic choices. >> i wanted to emphasize and add
i think the relationship is critical but also giving something back. it's been touched on a little bit in this meeting but not that much. it doesn't have to be a monetary gain that the patient gets but something that engages them with
their health and/or with their we've done this in a cohort of various insurance groups, so migrant farm workers up to high tech workers, 300,000 people, 82.5 opt-in rate for research, the retention rate was similar to that, actually over the course of a year.
it's really think establishing a relationship. we also looked at this in a group that we've coalesced with dia, that was partially government, european and american, partially industry, lots of foundations and some patient groups.
and there was one consensus that government and industry have a problem with the trust relationship so i think that's something we need to work on in terms of fielding a proposal to back to the patient that's acceptable to those different >> i think giving back is so
important, and we have resources that we don't even know are resources. and that we double give back. so when i work with many young girls, young girls of color, and having my students just help them with their homework, it helps the students, gives back
and we hear the stories. giving back to the population raises trust in the kids and in the parents, and we have a lot more things that we don't even think about that we can give back with, including their data. >> i'd like to just say one quick thing about trust.
our experience with the elderly, you remember the video, there was a little sensor that measured quality of sleep that was underneath the bed, that was the only thing in the video that isn't with a way. it was vetoed by the elderly they said don't spy on me.
there's an issue of trust for elderly people in a lot of ways. you're taking my money, trying to put me into a nursing home, trying to kill me. those are the three things that-- and as a result of that, you need to be really sensitive to
what elderly people are going through. you need to listen. we have really designed our system to respond to the issues they feel, so they can trust us. and we're just about ready to start moving into the sensor stuff for sleep but it's taken
two years. >> a few comments on quantitative aspects the nature of the data we gather and relevance of it, number one, we do have analytical tools that are powerful and diverse, someone asked are there tools. yes, there are tools and
nmr and mass spec-based with initiatives in canada and europe with creation of databases of knowledge on metabolism, all metabolites in different types of diets. number two, what is important in capturing the effect of the environment is not only what
we're exposed to and we only sense part of that, i would encourage we engage niehs that is thinking about exposome and implementing projects because what we are capturing from environmental exposure is a small piece of what's going to be captured with these powerful
analytical tools. number three is that the drugs are also critically important and impact metabolism and profiles and health in many ways, the effect of the drugs, we're mapping and creating catalogs of the effect of these medications so these types of
databases are going to be quite important and informative. >> on that note could you identify yourself and could all future speakers identify them 70s? >> rema douk from the duke medical center, so i think the-- one last comment, if i
may, also on the issues of mental health -- >> we've got to wrap up. next speaker patiently waiting. >> from the framingham heart study, we talked about trust and relationships, that's one of the strengths of framingham is that we've developed the trust
because we have significant relationships across generations, one of the things when you have this kind of trusting relationship, how easy it is to expand. so for instance when we brought in the generation three, we were actually funded by the nih to
bring in three thousand of the gen three, 6000 responded in the first recruitment, we went back and got in permission to bring in 4000. i can tell you right now our generation four is asking what about us, so it's a way once you develop it, the relationship, it
expands and that's where we talk about the family rather than the >> a follow-up question, just in framingham, where your patients are ascertained without perspective to disease necessarily, what are you giving back and what is framingham you view of giving data or other
compensation back to patients? so first of all we don't call them patients. we call them our participants and recognize they are volunteers. one of the first things we do at framingham is understand they are the most important.
i always tell my participants that as a researcher i'm a dime a dozen, the value of framingham is actually with them. one of the things we do is communicate constantly. we always give them extreme respect, no matter what. one of the things when i was
first brought into framingham, i was told clearly, you wait for them, they don't wait for you. so we carry that forward. we're very careful about how we handle our participants. we respect their time. we respect their contribution. and we try show that.
we don't pay them at all so this is all volunteer base but we do do things like we give them newsletters, heads up about things that are coming down the pike, and it does help. we tell them when publications are out there. we're in the media, that helps a
lot as well. it's a constant interaction with >> return of data, what we do in terms of-- we do inform them. if they would like us to send results to participants, we do-- to their physicians, we do do that. we do monitor them, so that if
we see anything, we do contact them and encourage them to go to their physician so we do sort of active monitoring as well as we don't give them data, per se, because we're worried about interpreting research-based data which we're not really sure about what these results may
mean, so we're comfortable sharing general information and then also sharing it with their physician. well, since there are no other questions i want to share a little experience. i talk about the ivory tower trying to reach the community, i
actually had experience coming from the ivory tower, trying to work in the community. i launched a study from the brigham in boss ona long time ago that was supposed to be launched in san francisco, it failed miserably. in my ivory tower at ucsf, san
francisco general, i was designed to study from the ivory tower and approach community, it didn't work. we revamped and it worked quite well when we recruited through qualified federal health centers and team members did the recruitment.
that's been my experience, recruiting 10,000 minority experience from throughout the united states, my experience. i would like to hear other people's experiences that have done similar work in the i don't want a topdown approach. i'd like to hear from people in
the community and brought one who runs the pediatric asthma clinic at san francisco general, primarily the safety net for all of san francisco, despite the fact that we have a huge wealth in san francisco, we have huge disparities, and i'm sorry to put you on the spot, miss kim
honda, who runs the clinic. would you able to provide practical experience you could share with us? >> hello. i'm kim honda. i do run the pediatric asthma clinic at san francisco general hospital, which is an
environment quite unique. if you've never been there, if you're not familiar with it, i think there have been such wonderful ideas generated here today, it's inspiring, it's always great to come up with challenges that surface as a result of these conversations
but the thing that continues to come up for me as i think about how we would bring these ideas back and implement them into our clinic in real life when we work with our patients is how do we really engage families and engage patients in a way that brings value for them.
we talked about different incentives and how monetary gain is not necessarily something that would fit for one population versus another. but it is really challenging to find something that would add value to a patient's time or family's time who is coming in
to give their information and sort of their health access information to you. >> we as providers, should we tell what you we new england you need-- what we think you need or tell you the research questions or have an organic abroach?
>> organic approach is great but it's been brought up in talks, talking with patients and families, discussing their aims, our aims are not necessarily in line with what patients are seeking when they come to clinics. >> next question.
>> how do you deal with the multi-generation of-- do you see a need to provide something different? >> since we're on air, i want to repeat the question. the question is how do we address the multi-generational nature, particularly with some
communities like the hispanic go ahead. >> i wish i had a simple answer for you. i think it's a really relevant question, especially as we look at the way our patients define it really is always-- in many cases, the vast majority define
family as multi-generational and enter generational. having an approach that incorporation different perspectives and language, literacy levels, are all i don't have a simple answer. i wish i did. >> does a provider or researcher
need to match the community they are trying to incorporate include? >> it was mentioned earlier that having really field experience or being sort of versed in the community and population and type of experience that you are actually trying to address is
very helpful and i think that's valuable. i think certainly having cultural and language appropriate services and approach is important. i know that families who come to us really trust us and we have the most success with families
who really feel like they can engage with our community health workers because these are people they may have seen or known who have lived in these communities and grown up in the same areas, know san francisco, speak the languages they may speak, have family dynamics or look or speak
or act in a way that feels comfortable or familiar and that's a big part of what makes our clinic work. i appreciate that you put you on the spot. >> you mentioned something else that's a huge-- that is the different-- hispanic is not one
thing, and we're doing a bunch of studies now in florida and california to try and sort this. i've been studying it since i immigrated back to the united states. there's going to be a venn diagram of things that work and things outside the venn diagram.
the child is the center of my universe, that radiates out, a great way to address the problem but as our technologies get more agile and we find we have great databases of what works for different people, i think we're going to be able not-- one size is not going to have to fit all
anymore. and i think that's the thrust of the future for this initiative. >> if i could add to that a little bit. for the technologists in the room, let me articulate why this matters. we've been trying for a long
time, we've presumed in the work that we've done that many of the folks in the medically vulnerable populations that find their ways to community health centers where we work, we've long assumed they actually cared about losing weighs. obesity ranks pretty low on
their health hierarchy, news to tracking their data, i happened to be a quantified self, i have a quantified self, but it turns out they don't enjoy that experience, just merely for the purposes of seeing numbers accrue, they are more interested in maintaining their shape, they
are not interested in losing a lot of weight, they are interested in staying where they are but they are extremely interested in reducing stress, extremely interested in sleeping better, feeling better, being better connected to their friends and their communities.
it wasn't until we took time to spend time in the community we were confronted, it changed our algorithms for providing a comment earlier about providing feedback, it's the right and ethical thing to do, i also think from a scientific perspective we know when we
don't provide feedback we don't maximize engagement. for intervention, that requires ongoing engagement or observational trials that aim to try to keep people involved for some period of time providing feedback is critical. if you provide the wrong
feedback, i've done that several times, you really risk sensitizing these feelings where mistrust. it's absolutely critical that one does the work on the front end to understand what folks in the community are dealing with so that you're designing
technology in a way that best meets the user's needs. >> i wanted to comment about a theme that's running through this that hasn't been made as explicit, behaviors and attitudes of researchers, understanding limitations and strengths of different times of
expertise on a team. you're asking for examples. i've had so many people come to me when i was in an institution, to tell me, the p.i. doesn't listen to me, they are hired as the recruitment person, they give advice. if the advice makes sense to the
p.i. they may listen. if it doesn't it is rejected, and they again to feel they are caught between actually put in a boggs where they may do harm to the community because they can't get their boss to take what they consider to be critical advice. i think there's going to be an
education or a paradigm change in terms of what types of expertise are needed for which purpose, in a team, and the lead scientist may only be the scientist in one aspect, there will be other people who know the science, especially crossing social class barriers because
the difference in having resources that are very limited is probably something that nobody in this room really understands very well because by virtue of being here we've crossed that and that is critical when you actually live and interact with people who
have to make decisions based on surviving and resources, but we need those folks in the cohort to teach us more about resiliency and survival than we would with a good cushion. >> that point was made clear last time we met by the presentation from the jackson
which i thought they did a very good job of being involved in the community at all levels. dr. platt, you had a question? >> first a comment. this is another tour de force i'm struggling to know which pieces of advice you've given can be applicable to a cohort of
a million people, because a million is a lot of people. and there might be a couple hundred or a few hundred dollars per person per year to build the cohort, maintain the cohort, learn things useful to the entire nation about it. like a caller to a radio
program, i would be happy to take your answers off the air. but i think it would be -- >> long time listener. >> just would be extremely helpful, not just from you but from all the people who have been sharing their wisdom with us to understand what pieces of
it are absolutely essential, if it's absolutely essential how do we apply it to a cohort that's a couple orders of magnitude than the ones in which you've done your learning. >> that's a great assignment. i think we should all give you that answer.
>> i'm not going to answer it right now. >> do i look crazy? take a crack at it. >> one crack. keep the simple. fundamentally important, if we're going to be dealing with a bunch of different populations
and with that many people, just don't get fancy. get down to the essence of what you're trying to learn and keep it very simple. >> i will try. i agree, i was going to say keep it simple as well. one thing, a refrain i've been
hearing, multiple overlapping cohorts is one recommendation. one group in which we're going to do granular highly precise measurement. i don't think our comments are actually-- i think our comments are consistent with that idea, if the bulk of the cohort is
going to be the masses of americans, if the goal is to really generalize, i think for the 80% or 90% among whom we're not doing the highly intensive measurement we should keep it very simple and we should be very user-centered. we should be asking only for the
kinds of things that we should be asking lots of self report questions, very simply, i'm a psychologist, i'll give you all 50 questions, we should probably ask two. i think we need to provide and identify ways of providing feedback, and that's the thing i
think is where i think there's going to be a lot of challenges. i think we tend to overestimate how much feed bank is necessary, for medically vulnerable populations to facilitate trust. the cardinal experience i've had with medically vulnerable populations is people don't ask
what they think very often, and my experience is when they enter into a trust relationship they become engaged dramatically because it's one of the first time anyone has taken an hour, three or four, five hours, i'm a psychologist, and spent time asking and struggling with the
kinds of issues that they are dealing with. to watch it happen is a beautiful thing. don't ask that many questions, you'll get the answers, but trying to find ways of providing feedback that activates people is not going to be that
challenge and won't require that we have a lot of sophistication. we have a speaker in the back. joyce tong, research at 23andme. i wanted to respond to share our experience in this. i think for us as researchers coming from like more of an academic research background and
sort of colliding with this consumer internet group that forms the other half of our company, i think learning from them we've come up with some ways to address this problem which is if you have hundreds of thousands of people and you're trying to do research with this
group in a cost effective way what can you do? a couple things we've learned are you really want to iterate, if i try a facebook post, google ad word, what is my cost per acquisition? it's a hundred dollars i try. using a teared iterative
approach you can figure out what is the best way to get to your particular target. one of the challenges that you face when you do this is that all the research has to be done under and irb, right? i think people are used to saying, well, this is what i'm
going to do, get it approved by the irb, in the protocol and you'll be stuck with it. the approach, we're going to try this, give yourselves a hundred options and use the one that works best and monitor how you spend your money, we've tired so far to really maximize value for
effort. >> one of the reasons i'm particularly bullish on academic mhealth, we've not done that and results are impressive. when we start iterating refigure the common rule, the potential for what we can do is
going to be dramatic. you're next. >> andrew from the icahn school of medicine at mount sinai. for dr. gustafson, adoption, people in rural pam, for software adoption use is i'm curious, if you're going to do a representative sample of
americans with a million or more people, how you think about the importance of physical proximity to actually building a community of people who can engage in providing the data, working with whatever software is required for that, what your thoughts would be, because it would seem
to be experience based on my experience with consumer software to date. >> i guess for me, our experience has been that it's especially if you're dealing with communication systems, discussion groups and things like that, almost an havening of
having people from different locations at once. for instance, we can put somebody who from the inner city of milwaukee on the same discussion group as a farmer from richland center, two very different populations, and since they can't see each other and
they don't know much about each other, their communication has some real positive aspects to i guess i'm not as maybe worried as i should be about the distance thing. it doesn't seem to have been an issue for us, maybe i misunderstood the question.
>> one of the lessons of, say, facebook is people need shared locations in mind but you may not need access, physical proximity to maximize adopt effect. we see that in our work where a lot of our actual intervention delivery is distal from the
health center but having the health center as a place of common physical reference is very useful. >> first of all, it's a great we should think about the answer and take that as an action item. one possible answer to your question is if i think about us
as welldoc, we ask an enormous amount of information from our patients and yet we find they give it to us. we ask ourselves, why? the analogy for me is you have a dry sponge in bucket of water, i can take the bucket and dump it on the sponge, the sponge will
absorb some, the sponge will spill. if i take the same amount and tailor the flow rate and position my water flow at different points in the sponge, the sponge will absorb quite a bit. maybe a lesson says oftentimes
we try to get things at once, it's what you said, let me ask my 50 questions, when patients and people will be happy to tell you things relevant at different points in the journey. maybe part says it's in the design, we should think about the design because we can
actually get volumes of data provided it's comfortable to give it to you in bit-sized chunks that make sense to them. >> i'd like to add to that what we can find with the new technologies that we're using now, a lot of work being done, not enough but we're moving
towards that, what's a meaningful moment. what's the right moment to ask a question, when you're available, when you're interested, what's the right moment to deliver an finding meaningful moments with new technologies is one of the most important endeavors we can
do. >> speaker on the left? joan broderick, usc. picking up on that, i think one-run of the things we're hearing is when patients or participants feel listened to, it's a very valued experience. to the extent our data
collection triggers meaningful interactions, so arthur stone talked about burst design so we're monitoring our participants and something happens where we want to learn more intensively about them and contact them to try to engage them in a detailed way, they
realize the data that we're collecting has meaning that we're monitoring it, that we are taking note of significant experiences, and that then we want to learn more about them. i think that's a very valued thing that doesn't require human engagement or telephone calls or
community meetings, although i think those are valuable too. >> wonderful. i'm going to wrap it up with a good example. we're on the precipice of a wonderful opportunity, the nih is trying to get this right, and making sure we solicit input
from all facets of america. we've rolled this out on the east coast, middle of america, now the west coast. it's important to make sure we have diversity of representation, that makes good social justice sense, good reason, but it's not enough.
and what we want to make sure, my goal, is to make sure we have scientific integrity, and the inclusion of diversified populations is important because we learn a lot about the biology of disease. you'll be impressed to know that last year at this time the
attorney general of hawaii sued the company bristol meyer squibb for marketing plavix, not knowing but it didn't work in 55% of asians because they didn't have the gene to metabolize, 55% chance of placebo, pacific islander 65%, highlighting the social reason
and scientific reason for diversity and by missing inclusion of diverse populations is as naive as not including women in clinical trials, or children for that case. having said that, i've done my best to bring kids in. i brought my 17-year-old
daughter here, a high school student. her first response, you medical times are way lined the user she told me siri didn't work for women. i want to thank the panel, i want to thank the moderators and the audience, particularly, for
sitting here for the last eight i want to thank intel for hosting this and eric being our point person here. for tomorrow, please keep your identifications to get into fort knox here, and we'll start here at 8:00 a.m. promptly tomorrow. any other announcement?
>> let me add one thing. tomorrow morning when you come in, you will not have to go through the main lobby, the bus will let you out, you come through a private entrance without the adventures of getting in, just to the right of where you came in tomorrow.
>> audience, we want nuts and bolts, we want your input.