[This Transcript is Unedited]
Hubert H. Humphrey Building
200 Independence Ave., SW
Washington, D.C.
Agenda Item: Introductions -- Review Agenda
DR. CARR: We now open the Data Access and Use Working Group. Keeping with our tradition, we want to start on time and move forward efficiently. What we would like to do is go around the room.
(Intro around table)
DR. CARR: Is there anybody who called in on the line?
Agenda Item: HHS Chief Technology Officer Perspectives
MR. SIVAK: Thank you very much for having me today. I just wanted to say, first of all, that I have been hearing some stories all day long about some crazy barbecue that happened last night. All I have to say is that I want an invite next time, so put me on the list.
(Laughter)
MR. SIVAK: Again, thank you for having me. I unfortunately have to keep it relatively short as a meeting with the deputy secretary at 2:30 today, that I need to be at. What I did want to do today is kind of go through a brief overview of what we are doing with Healthdata.gov, kind of what some of the directions we are heading in are, and kind of get a sense from you guys about how that fits in with the work that you are doing, suggestions, comments, thoughts, all of that kind of stuff.
I will just dive right in. As I am assuming most of you guys know, Healthdata.gov is sort of the version two of the platform that was launched with Data.gov, sort of way back in the day. We launched Healthdata.gov in June of this year, coincidentally with the third edition of the Health Data Initiative Forum, or Datapalooza as it is affectionately known.
Actually, since about a month or six weeks ago, we have been working on sort of phase two of version two. We are developing this with an agile team, using two week sprints to build new features and things like that. We are going to be releasing this version, this phase, on October 8th, next month. There are going to be a number of primarily back-end improvements to the system. I think the one that is most interesting for the folks here is that we are going to be improving the publishing process and the workflow associated with that, for the metadata associated with the datasets that people put out.
I think a message went out, actually, I know a message went out relatively recently to all of the health data leads within HHS. We are basically creating categories of folks who can go and author metadata entries and then edit metadata entries. It is actually kind of going back to the discussion that was just happening, it is relevant in many ways to that. We need to make sure that the datasets that we are putting out, now that we sort of have the control of Healthdata.gov and what datasets we put out there, we need to make sure that the datasets are both of high quality and sort of conform to the issues of the mosaic effect and re-identification and some of those challenges.
It is actually very important, but I think the new workflow that we are putting in place here will allow us to do that much more effectively. That is just one of the improvements. Keep that in mind.
For future iterations of Healthdata.gov, I am personally very interested in the feedback of both internal and external users, because I think we should use that to guide our development. Today might not be the right forum for this, but I would love to get input from you guys about the direction that you think this thing should take.
I really consider this a marketing tool for us. We have thousands of datasets across this agency or across the department that I think are incredibly valuable. In fact, it was funny, I got an email from somebody yesterday who had gotten the email about the new Healthdata.gov process, and said something to me about a couple of datasets that they have that they don't think are really that important. They are not really sure if it matters.
I looked at it and it is great data. It is data that you can think of a million different use cases for. I think there is an education process still that we need to go through internally. This site is primarily a tool for us to get this information out. The more usable and user friendly it is, the better, the better the descriptions of the data are, et cetera.
Really, I think over the course of the last few years, this has been, as you guys all know, a very intense, ongoing effort. These are some of the high level things that have happened with Healthdata.gov over the past really two and a half years, but primarily over the past, say, six to eight months as we have kind of built this out.
There is the metadata enhancement, or the publishing workflow efforts that we have been undergoing. If you guys have noticed, there's a blog on Healthdata.gov, which we've used pretty extensively to start to publish some good material on how data is being used, what some of these datasets are. I am a big fan of stories, so if any of you guys have stories that you want to promote, things that you want to tell, I think this is the way to engage people. We are happy to use this as a platform for anybody who wants to promote these things.
I should also say that we are looking at ways of enhancing the design. The look and feel, I think, can be brought up to data a little bit. I don't want to put anything negative on the team, because I don't think any of us are really designers. We kind of put this together, without having the expert services of somebody who is very familiar with design, and look and feel and user experience.
We are looking at ways now of incorporating that into future versions. We might even run a design challenge on Healthdata.gov, which I think would be kind of fun. That is all to say, marketing tool, if you guys have stories, bring them on. We are happy to use the platform to kind of publish those things.
I think a big step forward was providing programmatic access to the metadata catalog through APIs. That is really a first step. I think the real holy grail is providing an API for all of the data that we have, and that is obviously a much tougher lift, but something that is obviously right in line with the digital strategy, and everything that the entire federal government is kind of moving towards. I think we are seen, rightly so, as leaders in this space, and so we can continue this by moving aggressively down this path.
This next bullet point is a bit of a misnomer. One of the things that we are working on doing is enabling the ability to actually store data within Healthdata.gov. Right now, it is just a metadata catalog. That is great for the larger agencies and the folks who have some resources to actually host their datasets. There are lots and lots and lots of examples of smaller entities in HHS that don't have this ability. We think that it would be very useful to be able to provide them a place to maintain some of their content and information.
At the same time, using features of the platform to actually provide these APIs automatically on the data. This is something that we are working towards. With this new release, we will have at least the sort of basics of that capability. That is something to kind of keep in mind, going forward. If it is a heavy lift to create an API for a specific dataset, it might even be easier to bring that dataset into Healthdata.gov, and then use the tool itself to provide the API and the functionality that we need.
Then, finally, we are big fans of open source. It is the whole reason that we are building this on Secant and Drupal and SOLR. We are working now towards releasing the entire code base as a project under GET(?) in the next few weeks. We want to encourage development of this. Much like the data, we don't want to lock up the code within the walls of HHS, so we are trying to release this and get this out there as much as we possibly can.
One thing that has really kind of struck me about the whole data movement over the past three years, and I have been working on this stuff since I was with the DC government a few years ago, is that we have had this big push to get data out. We haven't done a ton in terms of helping people understand what the data is all about.
I did this before, I will do this here because I think it is kind of fun. Just as a really simple example of a dataset that we have, that I think is very valuable, but it is not accessible to normal people, is one called the National Plan and Provider Enumeration System Downloadable File. You guys all know what this is? I figured everybody here would know what that is.
Pretend that you don't know what it is for a second, and listen to the first sentence of the description of this dataset. The National Plan and Provider Enumeration System, NPPES, Downloadable File, also referred to as the NPI Downloadable File, contains FOIA-disclosable NPPES health care provider data for health care providers who have been assigned national provider identifiers, or NPIs.
I will admit, I read that and actually one of my guys that you all probably know, Arnum Chatagy(?), was showing me how we had included this dataset as part of our health data initiative starter kit, and how it is a great dataset. I read the description and I was like, I have no idea what that means. I said, what is that, and he said, oh, well, it's not totally complete, but it's pretty much a list of all of the doctors in the United States. Why didn't we say that? That's a useful dataset, right?
Every time I have said this to an audience that is not primarily composed of subject matter experts, their reaction is, wow, that is useful. We could use a list of all of the doctors in the United States, along with their location and specialty and all of that kind of stuff.
To me, that's a perfect example of some of the work we need to do, in terms of really broadening the tent and talking to a much wider group of people. I think there is a lot of talent out there that we can take advantage of, that we are missing right now. One of the efforts that we are going to be working on is, for lack of a better term at the moment, call it data education. We want to basically use the skills that we have internally, to translate things and make them available to people on a broader basis.
That is kind of one part of that. Another part of that is we have got people here who have been working on these datasets for a long, long time. Up until now, they really haven't had much contact, in some cases, with the people who actually could use the data, or maybe who aren't using the data, but would like to. I think that there is some value in that two-way conversation, both from the perspective of the users of the data, who might not know what it means?
Right, I mean, if we could have the person who creates the NPPS or maintains it say, hey, it is a list of all of the doctors, and here is how we collect it and here is how you use it. That would be helpful to people. At the same time, I think it would be helpful to the people inside, who are working on this stuff, to see how people on the outside want to use it, how they can use it, how they are using it. I think that will help influence that we develop things going forward. We are making an effort now to connect people, both virtually online, but also in person, and I will talk about that in a second, to kind of help spur some of these developments.
Finally, something else that really struck me as being a little strange was that we have our Healthdata.gov project, which is based on Drupal and Secant and SOLR, going on. Simultaneously, as the other federal government entities have this thing called the OGPL going on, which is based on Secant and Drupal and SOLR. We are both doing it because they are open source projects, and because we want to be able to kind of control the development of these things, or at least control what features and functionality we think are important, we put our resources to. We want to own the code. We want to build communities around this stuff.
Right now, we are building two separate communities, which really doesn't make a lot of sense to me, because it is on the same stuff. We have recently started a process to get everybody together and start working on the same code base, to kind of put these projects together. Obviously, we will have different features and functionality that we prioritize, but that is fine. That is how open source works, right?
I just feel like if we put everybody together and get a single code base going, that is this project, we will have much, much greater buy-in from the rest of the world. We will have a virtually unlimited pool of developers to tap into, that hopefully can expand the scope of this rapidly. So far, this process has actually been working pretty well.
It looks like there is a lot of international interest from the folks in India and the U.K. and Canada. The U.S.'s been really psyched about this. The guys in the Healthdata.gov team are pretty excited about this, so I think this can go somewhere. For any of you guys who want to pitch in on this effort, I think it would be great. The more support we can show for a unified band of people, kind of working together on this thing, would be really helpful.
Let's see, I mentioned the memo that we sent out to the health data leads already. If anybody has any questions about that, feel free to come talk to us at any time. The basic timeline is pretty straightforward. We want the leads to identify their internal teams, the authors and the editors, by October 1st. We want those leads to then compile a catalog of just what datasets that we are talking about by the 10th.
Then, we are going to do a training web to kind of walk through how the system actually works and what the new functionality looks like, on the 10th, as well. Then, we will hold office hours until December 1st, to kind of answer any questions that come up. This is the general idea, that we want to get across.
Now, in that webinar on the 10th, one of the things we are going to be doing is refreshing people's familiarity with the privacy issues and the data quality issues. I think that previous discussion, a lot of that stuff is probably relevant. It is probably worth getting our guys in touch with some of the folks here, before that happens, in order to make sure we have the current thinking kind of in place.
I think the other thing I should say is that over the past few years, we have spent a lot of time focusing on the idea of changing the culture at HHS from this default setting of closed data, to a default setting of open, with thinking about what to do after, as opposed to a blanket restriction and then trying to open it. I think that has been remarkably successful. I just showed up here a couple of months ago, and I can tell you, just based on my conversations, it is almost a presumption of openness now among most people. I think we have done a great job there.
There are sort of pockets of resistance, I would say, within the department that we still want to talk to and get them sort of onboard with this whole thing. We are undertaking this process of meeting with the data owners at the agencies, with everybody at pretty much every level, to kind of again explain the importance of this, walk them through what we have done over the past few years, talk about some of the challenges, some of the successes, make sure they understand that this isn't really a terrible thing and there are lots of good things that can happen from it. Then, also, get feedback from them about what some of their issues are, how we can help resolve them, what we need to do in order to make this work better.
We started with the Administration for Children and Families. We met with them in July. That meeting actually led directly to a whole bunch of data being released just a few weeks ago, which is incredibly useful actually for many, many, many purposes. As part of this effort of getting the people who know the data together with the people who can use the data, we partnered up with some folks at the Greater Baltimore Technology Council, who are running a set of events called Unwired and then Groundwork.
Unwired was the first one, and the idea was to basically define the problems that existed in the city of Baltimore, figure out what assets were available to solve those problems, have everybody form into teams around specific projects designed to solve those problems. Then, the Groundwork event, which happens next week on the 28th and 29th, is to actually build those solutions, with the time being spent in those intervening weeks to actually kind of add more assets to the projects and things like that.
It turns out a lot of the ACF data is useful for some of those projects. We had people at the first event, at Unwired, from ACF and from SAMHSA, who were there to actually provide their expertise. They are going to come back to the events on Groundwork, be part of the teams and help explain what data is out there, how it works, all of that kind of stuff. We are looking forward to seeing how that goes.
It is an experiment for us. Even if it is not successful, I think we will learn from it. I would like to see us try to replicate this in other places. We have already had some interest expressed from folks in Philly, from some folks in Boston, some West Coast cities. I think there is this really interesting idea of connecting people who understand the information with people on the ground who know the problems, and have the access to other assets, to try to mash it all up and do some interesting things with it. This is another thing that we are working on.
Finally, we are having our biannual meeting for the HDI data leads in November. We will send out more information on that, so stay tuned for that. Almost finally, we are working on a bunch of different things externally.
We have this regional affiliates program for the health data initiative, the idea being that there are lots of interested parties around the country, who want to be a part of this and kind of do their own things in their regions. We are encouraging as much of that as we possibly can. We have some great examples here. In Cincinnati, two weeks ago, there was an event focused around a health company accelerator that had some really interesting applications come out of it.
There is an event, the first week in October, in Boston, a Massachusetts regional affiliate, kind of along the same lines. There has been a bunch of activity out there, and we are encouraging more and more folks to kind of sign up for this program, that are interested. Our job partly is to support those efforts, and to help those people kind of do as much as they possibly can. I should say you can find more details on this at the URL at the bottom of the slide, hdiforum.org. Again, if you have any questions, feel free to contact me.
Finally, the event that you all have been waiting for, the Fourth Edition of the Health Data Initiative Forum, or Datapalooza. Save the dates were just announced, June 3rd and 4th, next year. Mark your calendars, don't plan anything else. Be there the whole time, it will be fun. We are in the planning process now, so I would, over the next few months, just keep an eye out for lots of different pieces of information coming through around schedules and agendas and suggested topics. If you guys have any thoughts, let us have it. We are definitely open to suggestions. That is about it. I am happy to answer any questions. I think I can stay for one minute.
DR. COHEN: If data are going to be housed in Healthdata.gov, how is that going to interact with the health information warehouse?
MR. SIVAK: That is a good question. My personal opinion is that the datasets are slightly different, or they can have slightly different purposes. It is an alternative, in the sense that there is a different use case for it, maybe. The Health Indicators Warehouse, there is a lot of analysis that goes in there, and a lot of sort of aggregated data and things like that. I am imagining that, if we want to provide programmatic access to raw datasets that don't currently have a home, it is not a bad place to do it. These are all things that can be worked out. This is an initial stab.
DR. COHEN: Is the focus going to be on individual level or aggregate level or whatever is around?
MR. SIVAK: I think it remains to be seen. I would say that kind of whatever is around would be my answer right now, but that is something that we can talk about.
DR. WARREN: I love the fact that you use the NPPES as an example, because you described it wrong. When we are using these datasets, we need to be very careful, if we start changing the wording, that we have accurate descriptions. It is more than just physicians, it is any clinician that needs an identifier in order to be out there.
MR. SIVAK: I want to ask you a question about this. That is a great point, and this is something that I have seen kind of throughout my career, when talking to any subject matter expert in any field. Actually, my best friend in the world, actually he is a philosopher, right, I used him as an example of this.
I went to this thesis defense because I wanted to be a good friend and show support. I sat down and it was a four-hour long affair, with him and the committee that was sort of integrating him. They spent four hours talking about his thesis, using words that I understood, I had heard before, they were part of my vocabulary. They were using these words in combinations that literally made no sense to me. I understood maybe 15 percent of his defense. I like to think I am a pretty sharp guy.
What I realized at that moment was that, in technical fields, we have a language that we use because we have to. We have to be precise and we have to use very specific words, in order to communicate what we are trying to say. At the same time, if we are trying to involve a broader group of people than we typically talk to in our technical fields, I think we have to give up some of that precision, in order to be more approachable. I think this is a potentially interesting example of that, right?
DR. WARREN: As long as we don't disenfranchise other people.
MR. SIVAK: Sure, and absolutely right, but I think there is a way to phrase things in sort of lay language that might not be as precise as using the word, provider, right, but might be more accessible to folks. My gut tells me that we can involve a lot more people if we do that. It is something that I am happy to have the conversation about, because I am interested in this quite a bit.
MR. SCANLON: Bryan, just before you go, this is the group we have just formed. We have folks who know the data area, the public health area, the community health area and the technology area, and folks who have helped plan previous Datapaloozas. We will be using them, and it is available to your office, as well, as resource experts on how we are doing with HHS data. Do you have ideas about how we can reach out better to the developer community, and what kinds of datasets do you think would be more useful.
The other thing is, as a FACA, you can hold meetings with the public, and with anyone really, any group, and be covered under agency consultation. Again, it provides the opportunity for open meetings with these communities for the department, as well.
MR. SIVAK: That is great. I am sorry I have to run. If anybody has any questions, please feel free to just drop me an email. It is just Bryan.Sivak@HHS.gov and I am happy to help.
DR. CARR: Thank you so much, this is great. It was clear and we understood it. All right. Now, Jim, I keep preempting you, but back to you, review charge and agenda.
MR. SCANLON: To review, we invited you, you were all recommended highly to serve on this working group. The focus really is, as Bryan said, we are not developers. We have a lot of folks who are experts in data and surveys and research and programs, but we collect a lot of data. Some of it is intended to be public, and they have extensive dissemination programs. Other data is more administrative and you sort of have to turn it into a data product.
What we are going to try to do, and we really look to you for help, what would be the best way. We would like to expose you to the kind of data that HHS. To be honest, what you see on Healthdata.gov is probably the tip of the iceberg. That is the data that can be put on an open health data website with no restrictions. I think you know, that is probably not what we can do most of the time, other than directory data or location-based data, public lead data.
We have a lot of other data. We have all of the CMS claims data, for example, that can be made available for research analysis quality and so on. We can't put that, for the reasons you heard at the previous discussion. We can't just put that on a public website and cross our fingers. We will all be in jail before the day ended.
What we are asking is your help on what data do we have. We will talk about how we make it available now. If you could advise us on how we would reach out even further, I think you particularly could develop a community. How would we even interact more, because again, our folks sort of stop when they publish or move on with the next study. You really know how that data is taken and applied. Many of you make a living doing that actually, so that is the kind of advice that we would need.
All the while, thinking of protecting where we are making it available, how do we be sure to protect the confidentiality, and generally give us advice. I think we will start, as we will today, with two of our biggest data holder producer organizations, the National Center for Health Statistics and CMS. We will take you through some of the others, as well, at Public Health Data, and we will be looking for more advice.
DR. CARR: I would just add, I realize this is our second meeting, and Todd really asked us to work quickly, and we took the summer off. It is our intention to be nimble, quick, focused and really take this. In fact, Bruce brought forward an excellent application where we may begin to say, here is what is out there, and here would be a way to use it. We will talk about that later. I think we are going to hit the ground, working hard, before we leave here today. With that, Jim, can we turn it over to you?
MR. CRAVER: Thank you. I was hoping to squeeze another 10 minutes out of your agenda, but that has been taken from me.
DR. CARR: No, we are flexible. We are here to learn basically.
Agenda Item: Overview of NCHS Data Products and Services
MR. CRAVER: I am here on behalf of the National Center for Health Statistics. Really, my objective is to give you a whirlwind tour of some of the pieces that are available to you. I am going to try to jump back and forth, between the Web and the presentation. If I am going too quickly, stop me.
DR. CARR: Did you say the Web?
MR. CRAVER: Yes.
DR. CARR: Are you familiar with the HHS building?
MR. CRAVER: Yes, I am, so I can stay within the presentation, if I have to. Thank you for the reminder. These are some of the areas that I am going to touch for the presentation, just a brief overview of some of the data systems and how we think of them, how we organize them. Then, ask the question or answer the question, why might we approach a single health topic from multiple perspectives with multiple surveys. Then, dive in to some of the tools and some of the resources that are available to you and to the public really.
Again, my objective is that so your familiarity with NCHS and its resources is increased by the end of this talk. Also, for you to be empowered and able on your own, to go even more in-depth into some of those resources.
Just broadly speaking, the types of data sources that we usually refer to within NCHS are the vital statistic system, the births and deaths, mortality and natality datasets. We have surveys of individual people, and that includes person to person interviews, computer-assisted interviews, knocking on doors or telephone surveys, as well as bringing people through our Mobile Examination Clinic, the MEC, which not too long ago was sitting out in front of this building.
MS. GREENBERG: It still is.
MR. CRAVER: It is still here? I came in the back way.
MS. GREENBERG: We had hoped to visit it during this meeting, but there was just too much going on.
MR. CRAVER: That is a lost opportunity or missed opportunity, too bad. It is maybe four and a half trailers, I know they say five, but the last one is a small, short trailer. For our new survey, the National Youth Fitness Survey, which is bringing in youth, young children, and adolescents, and actually measuring their fitness, and their height and weight. It is really the gold standard of clinical measures of individuals.
You can imagine the provocative statement that I always say is, imagine you walk up to someone and ask them what their weight is and you will get a number. If you put a scale on the ground in front of them and ask them to step on it, you will get a different number. You get a sense of kind of the reasons why we like to approach topics from multiple perspectives.
We also have the National Health Care Survey. It is a family of surveys which do survey providers, doctors, clinicians and others, who are in the health care arena. That is really quite a broad area of surveys that are coordinated and ongoing. Those tend to be of administrative records or records that are abstracted from hospitals and clinics and other places where people interact with the health care system.
I have sort of hinted at or touched on some of the reasons why we have multiple sources. You have probably have guessed already, to really capture all aspects or all facets or all sides of the health care industry, the health care system, health care as it is used, and health care as it not used. We even have estimates of undiagnosed disease prevalence and incidence rates.
We also use our data to look at some of the methodological issues regarding collecting the data and analysis of the data, how best to understand the data that we have, how best to combine that data with other datasets. Also, to look at extending a data system that we have through linking it at the record level to other datasets. We have an ongoing program, we call it the Linkage Program, that takes Social Security Administration data, and takes our survey data, and it takes CMS data and census data, and we merge that together at the individual level, at the record level. We essentially come up with another dataset that really has a full range or a fuller range of information at the record level.
Your discussion about disclosure and re-identification starts to make some of the people at NCHS very nervous, because we do have this kind of data that is tapping into data sources from multiple agencies, and really intentionally doing that. Then, we have to release that data in an aggregate form, in a form that is perturbed in some way, or guards against that disclosure. Or provide access to researchers to the raw data that we have, in a very secure format, with high levels of assurance from them that they won't do anything that they shouldn't do with those data.
These are the tools. I am going to get up and grab my water, because I am starting to get dry. It is warm in here for me. These are some of the tools that I will touch on. I will try to jump over and take a look at those. If I don't succeed, because it will actually take me a little bit longer if I do that, but if I don't succeed, I will stay within the presentation.
Just looking at the homepage for NCHS, I will give you a little bit of the geography. This is maybe a month and a half old. Our website was revamped. We are very proud of the work that went into this. I have to tell you, though, as an older user of NCHS website, it took me about three days and a phone call to find out where FastStats was. There is a big red arrow, that is to help you. It was right there in front of me, and I couldn't find it. The person who I called was very nice and didn't say anything bad.
Scrolling down the screen, you will get to the data access areas, and then you will get to the additional resource areas. Obviously, this page has a lot of information and a lot of links that go off into different parts of NCHS. These five red arrows point to the five areas that I am going to look at today.
The first is the Health Indicators Warehouse, the first one that I am going to feature. I appreciate question earlier, Bruce. It is one that I think about often. Bryan and I have talked, and we will have ongoing conversations about it, so I have my thoughts that are not inconsistent with some of his thoughts.
The Health Indicators Warehouse is all aggregate, public use data. There are no individual records, it is all data that anyone can come and use, access, download and access through an API. The system has the ability to graph data. There is some linking to interventions. We use, right now, the community guide to preventive practice and the guide to clinical preventive practice for some of those. It was an idea that was proposed very early on, that we don't just put out indicators like the number of people who have diabetes. If someone in a community wants to do something about that, how can we point them in a direction that they then might be able to do something about.
Because that data is available through RAPI, that then can be used by third-party developers, which is really one of the things that this project is really trying to drive also, not just open data, but also to seed that community of developers, third-party developers, who aren't necessarily in public health, population health circles, but are developers, and want access to high quality data.
We have about 1100, I think it's 1170, 1169, indicators on the system right now. It comes from a variety of data sources. I am embarrassed slightly that CMS is not called out specifically here. It is included, of course, under HHS. We do have maybe about 150, 170 indicators that are from CMS, and many of those are only available through the health indicators warehouse.
Currently, those indicators only have a single year of data, 2008. We have, and are about to make available to the public, four years of data, and then, very soon after that, five years of data. There is going to be a little bit of a tweak for the interface, but that will happen quite shortly.
Let me get to the homepage and see if I can come over. Here, I am on the browser, and I am on our homepage at NCHS, and I am going to dive down the Health Indicators Warehouse. Just a quick geography, we have three main areas by topic, by geography or initiative, to get into the warehouse. I like to say that these are basically three doors into the same very large room.
What a user will do is select a topic and jump over to what we call our filter page. Immediately, you are presented with a subset of the 1100 indicators, related to the topic that you are interested in. I am just going to pick something. By the end of the day, this might be relevant. I am looking at binge drinking in adults.
We don't force you to read the metadata, but we do force you to click through the page that has the metadata on it. At NCHS, we do think that that is very important. You don't really know what indicator it is you are looking at, just by the title of that indicator. You need to know something about the methodology, something about the numerator, something about the denominator. We, at least, put that in front of you.
We also tell you the data source and what years might be available for these data, and some of the dimensions or variables that we have. It is a quick click over to the data display, which we start with a chart. In three or four clicks, you are looking at the data that we have on a particular indicator. This interface is to help you get familiar with the content of the warehouse. We then let you download these data, on an indicator by indicator basis. We also expose the database through an API, so you can gather as much about all of the indicators as you want.
DR. TANG: Who determines the 1100 indicators the first time, and who determines that HHS will actually maintain it?
MR. CRAVER: A really good question, it is one that we struggle with. We started with what we refer to as an initiative. We made the assumptions that there are groups out there, within HHS and closely aligned with HHS, that have already done that heavy lifting. The obvious one for us is Healthy People. The data stewards for Healthy People happen to be in NCHS. They happen to be about 50 feet down the hall from my office. It has been a very nice collaboration.
Now, ODPHP and assistant secretary for health, they have their interests and their agenda, and their audiences for Healthy People. They also have a heavy respect for the data that go into that. Right now, they are well over 50 percent of the indicators in the warehouse are associated with that initiative. We also have a subset, not a complete set, but a subset of the county health status indicators, and the county health rankings of the community health status indicators, as well, as I mentioned earlier, the CMS indicators. Together, that adds up to 1100, 1200.
Moving forward, it is a very good question, because what we like to do is receive data already processed for us. We do some quality control check, quality assurance checks on that. Within this project, we do some data programming, also.
We try to increase the efficiencies with the other projects that are happening at NCHS, including Health U.S., which I will talk about in a second, which programs a lot of data to make available to the world, Health Data Interactive, which I will also talk about, programs a lot of data, makes it available to the world. But it makes it available in a different way. Both of those projects and products have their own history and their audience and their stakeholders.
We think it is reasonable, and we do have a governance body that is the overseers come from the NCHS Board of Scientific Counselors. Then, we have our indicator advisory group, which is made up of members, of representatives of different HHS agencies. We have a statistical standards group, which is made up of staff of NCHS, and representatives of some of the data sources.
Just to make sure, we have a couple of levels of criteria before we put any indicator and any data in here. For example, if you have a survey, we want to know what the response rate is and we want to publish that. We don't want to just say, it has a good response rate. We want to include confidence intervals where it is relevant, or standard areas where available, those sorts of things. We make sure that that is true.
Each year, when a survey repopulates its publically available dataset, then we go through and we process it as quickly as we can. It is not small task, as you can imagine.
I have jumped back to my presentation. Just to move along a little bit here, a couple of screen shots --metadata from a different indicator, stroke deaths. Here is just an example of a different way that you can display the data. We saw the chart format for binge drinking. Those data are also available in map format, state by state. Where available, we have county level data, and that is mappable, as well. As you can guess, most people want the lowest level of geography as possible, for as much data as possible. We assure you we do that. Unfortunately, that mainly means county level.
I am going to move over to Health United States, and I see that I am going long here. I am going to stay within the presentation here. Health United States, I actually meant to bring the volume. Who has seen or who owns a copy of Health United States? Everyone here should have it.
MS. GREENBERG: It is routinely sent to the members of the National Committee. Actually, for comment, but also, we will include the workgroup the next time it goes out.
MR. CRAVER: I think the first time the workgroup met, we distributed the Health US in brief.
MS. GREENBERG: If anyone doesn't have it, let us know.
MR. CRAVER: You are very familiar with Health United States, the reason for its being, the fact that it is legislatively mandated, and that it covers the broad range of sources and topics. It provides, for example, this window onto the world, how might you start to describe selected trends in health care use? There are tables on preventive services, prescription drug use and inpatient surgery. Here, they are associated with the different surveys that we use as the sources. I just have some example slides, pulled from the text of Health United States.
One of the interesting developments just around the corner for Health United States is a collaboration that we have with elaborate(?) medicine, to make an interactive version of the Health United States available to the world. You can dive deeper and deeper, and deeper still, into these tables, deeper than what is in the published version, deeper than what is on the Excel sheets that you can download for the content of the warehouse. That might deserve its own presentation at some point.
I mentioned Healthy People earlier. We have recently transitioned from Healthy People 2010 to Healthy People 2020. That allowed an opportunity for people to review the indicators that are there, the number of objectives have increased. That, I think, just reflects the additional feedback and visibility to that project. Moving forward, their challenge will be to track that data and to keep that data updated. One of the things that they have done is to hone in what they are calling their 10 leading health indicators. They are producing reports about those 10 leading health indicators for Healthy People.
The example that I have pulled out here has to do with heart disease and stroke. For Healthy People 2010, there were 17 objectives tracked. Now, there are 18. There were five data sources. The same data sources are used, but there are an additional 31 developmental indictors. Developmental indicators are, if you want to think about sort of aspirational indicators. They are indicators or objectives that ought to be collected and ought to be available in order to more thoroughly understand the area or the topic.
This is a capture taken of the final review for Healthy People 2010. I think it is really worth looking at that PDF. The URL is up here, and I apologize that I did not have the slides to distribute. If that is necessary or desired, I would be happy to do that.
MS. GREENBERG: We will post them. Is the working group on the SharePoint, too? We will post them publically, because it is part of the public meeting. We can also put them on the SharePoint.
MR. CRAVER: I am sorry you don't have them in front of you. This is, I think, a really creative way to display a lot of information and to look at the trends on several indicators or objectives for a topic area, all at once on one page. This really does capture a lot of complexity. You can stay at that surface level, or you can look at the details and really gather a lot of information.
Health Data Interactive is another project that NCHS has. It also has aggregate data. It also has indicators, or in this project, we talk about tables. They are tabular views of the data. I am going to hold this up as I am talking about this, because it really has an interactivity that I think is worth understanding. Once you understand that interactivity, then you can really see the value of this site and why it exists separate from the warehouse.
When you drop into HDI, as we call it, it was the original HDI, I will have you know, it is not that Health Data Initiative, we had the name first. You come to our splash page with the table topics. When you click on a table, this screenshot shows the results if you had searched for the word, asthma, you get the list of tables inside the application. From there, you can look at charts, you can look at tables.
It is much more interesting if we look at, let's say, okay, ED visits in the United States. What I want to show you is this is our opening view for this table. We think that this probably meets most of the needs for most of the people who come to this page. However, these little shaded parts allow you to click and drag, and customize the view of this table. Suddenly, you have taken this dimensional hyper cube of data and twisted it, and shown a different face that you might be interested in.
Perhaps, you want to look at diagnosis by regions of the country. If that is not how you want to display it, then maybe this is how you want to display it. HDI, Health Data Interactive, lets you take these 50 odd tables, and not just look at the data that is there, but start to manipulate it and start to come up with a way of changing what you see, being able to focus on something that you are interested in exclusively.
Now, this is really your table. This is your customized view of the table. Of course, this can be looked at in chart format and where available by geographically, a map display. That is Health Data Interactive. That process is the same process that was used to create this customized chart. It really is an interactive tool for manipulating the data.
Lastly, I am going to mention FastStats. FastStats is the place to go, if you don't know the number, but you know the descriptor of the number. Say you want to look up the number of people who have diabetes and what is the current value of that. FastStats is the place to go. That is in the upper right quadrant of the homepage. It provides a quick access to a long laundry list of topics that we maintain and update. Whenever there is new data on a topic, we go in and we update that FastStats page. Here is an example page for diabetes. We cover that for, in this case, morbidity and health care use and so on.
That is the end of my talk. I encourage you to take a look at each of these tools, to explore the NCHS webpage. That is really your portal to these tools. Each of them has an approach and an audience and a reason for being, that is slightly different from the other. We are, at the backend, really trying to increase our production efficiencies.
One of the other things that certainly the Health Indicators Warehouses spurred us to do even more than we already is to pay attention to harmonization and pay attention to standardization. That is flowing back and forth, as we speak, with the Healthy People project, with Health Data Interactive, with CMS. It is an exciting set of tools that we have. I hope I have exposed you to some of those, and that you are able to take them and run with them. Any questions?
DR. CARR: I just want to say, it is great. It really is well thought through. It is sort of user friendly, because it makes sense where you want to go and where you find it. I really like it.
DR. ROSENTHAL: One of the external developers that Bryan and company are trying to target, what has been their reaction to it. Speaking as one of those, one thing that springs immediately to mind would be entity diagram. My question is, in terms of target groups, you mentioned it was for different audiences, different views.
One of my questions was, how do you reconcile this with what Bryan was saying, in terms of what is the nature of the reaction of external developers. From an external development kind of perspective, when I am looking at things outside health data, the very first thing I would expect to see is a big ole entity diagram up in the middle of it.
I am looking at interactive tables. If I want to develop, it is not enough to know this metadata is defined in sentences. I need to see what is this thing, and this goes for CMS, too, parent, county, org, et cetera. What is this thing, this piece of metadata, where does it exist as a piece of data or as a summary. It is very, very important for someone coming in, who doesn't understand the nature of the difference between physician, provider, clinician, et cetera, et cetera.
Show me that physically, what is the nature of that relationship. Typically, call it an entity diagram. We have spoken in other committees with some of the NORC and IMPACT people. They said there is no reason not to share that.
If I am coming in and trying to put on HHS's glasses, and take a look at what the world looks like through your eyes, when you're looking at your data, what is the single quickest thing I can see, I immediately go to that. My question is, this is all fantastic and wonderful, but from a development point of view, that would be the absolute first thing.
MR. CRAVER: I don't disagree. The warehouse is, to date, the data that is available through an API, and that is the place where there is the most critical need for an ERD. One of the issues that we have as a federal project is security. We have some people who are at one end of the continuum of locking down everything, and we have other people on the other end who say, it is public use data and there are no secrets.
For systems like that, we have to make sure that we are not handing the tools to someone who wants to crack it and open it up, and do something nefarious. I know that same continuum of people who anticipate that and worry about that, and make me as a project officer and project manager behave a certain because of their fears, whether they are real or not, they are what I am compelled to follow.
We do have plans to put up an ERD that should be sufficient. I do want to engage on a one-to-one basis with people who are trying to use our API, so that I can learn from them how to improve it, and what other resources. We do have user's guides and we have data dictionaries and those sorts of things.
There is at least one piece missing, and it may be just the ERD, but I think there is another piece missing, too. I actually have a person on staff, who I am working with, I basically said, you go use the API and you build me an example that I can post, an example app. I am circling back around with him in the next couple of weeks.
DR. ROSENTHAL: I have actually prepared some slides, and when we can get into what we could do, showing tangible examples of how to do this, especially with things that are very concrete, specific examples that have very little to do with security, in terms of nature of relationship between parent or contract. I can share those with the committee.
MR. SCANLON: Jim, if I am remember correctly, for the warehouse, anyway, the backup data that supports the graphics and others, isn't it machine readable? Am I thinking of Health Data? So that if you had the metadata, you saw an application, we have access to the data, as well, right?
MR. CRAVER: Absolutely, that is correct. What people have struggled with is, I mean, it is an obvious problem. You have a series of tables that are related to each other, through foreign keys, and you don't have the Rosetta Stone of how those relationships are built. If you know the data, you can sort of guess. If you don't know the data, you are lost in the wilderness. What we want to do is we want to encourage use, not discourage.
MR. DAVENHALL: Is it possible, Jim, that you can start to provide for this working group visitor statistics, metrics sites, give us some sort of sense of both anything else you can tell us about that.
MR. CRAVER: I can give you a kind of up to date statistics on the warehouse. We are generally around 1000 unique users in a given week. That has been pretty steady, it falls off in the summer, as you would expect. It is starting to ramp up again now that schools are back in session, people are back from vacation, so forth.
I haven't looked at HDI recently. That, a while ago, was more around the 5000. I am guessing that has tailed off a little bit with some of these other tools. Frankly, Health US, I don't really know. I could probably find that out, but I would have to look at that, and FastStats, also.
DR. CARR: The users are identified by their email, is that how?
MR. CRAVER: Again, there is something called CNA, Certification Accreditation, which any IT project in the government is supposed to go through. There are levels of oversight involved. If you collect PII information from visitors, such as their email address and their name, first name, last name, location, phone numbers, those sorts of things, you have a much, much higher hurdle to jump over, just to get your project out the door.
Most projects, including these, basically it is an open door and anyone can come and go as they please, and we don't track who you are. We might track whether someone has walked through the turnstile, but we don't take the fingerprint when they go through. We can give numbers. For the warehouse, we are currently using Google Analytics, which allows you to take a look at unique visitors during that time period that you are looking at. You can say, in a given week, this person only came once. Or if they came again, I didn't count them a second time.
The other tools, I am less familiar with how they track their users. Unless you see a site where you have to register or you have to deposit your ID, your email address or something like that, you are not likely to do that. Now, I am on the brink of making a decision, and having a discussion about asking more information about users of the API, so that we can provide more one-on-one relationship. I think it is worth the cost for me, as a project officer, to get that through. I think the audience will give that stuff away. They are not going to care about it.
MR. SCANLON: But you do accept suggestions or complaints?
MR. CRAVER: Absolutely.
DR. VAUGHAN: Are you thinking along the lines of what they are asking for labor, in terms of applying for developer keys?
MR. CRAVER: Yes, just so that we can then go back to them and get the kinds of stories that Bryan is asking for. Or say, we have this new resource, or we are thinking of doing this, can you give us some more targeted feedback.
DR. VAUGHAN: I was real interested to know, to what extent exists now, or is anticipated alignment of these same initiatives with what is going on in the states and counties, many of whom are also looking to open up their data, and kind of piggybacking on what Bryan is saying. They are using Drupal, blah blah blah blah. What does that alignment landscape look like?
MR. CRAVER: Well, the warehouse is not open source. Health United States, the data is available through Excel. HDI, Health Data Interactive is also in a propriety system, and FastStats is html, it is up on the web. We have not had much focused interaction with representatives from the states, although we are also on the brink of doing an evaluation study and trying to get input from state directors of public health and their deputies, feedback and baseline awareness on survey. That should be happening soon.
I am hoping that, in addition to getting that data, that will sort of climb a little bit of the pump for that back and forth communication, so we can have those exchanges. Now, in terms of open source, that was a decision to go down, not to have open source. It was a decision that was made, but that doesn't mean that couldn't be changed at some point in the future.
DR. COHEN: Can I respond a little to your question? There are, I would say, between 20 and 30 state-based, web-based query systems, either directly from state health departments or in conjunction with partners. Some of them are open source codes, some of them are in-house development, and some of them use essentially cut solutions. Then, there a couple of vendors who sell essentially products to create that functionality at the state.
Obviously, the states in general go down to county, and some go down below county to large communities. In a couple of cases, ZIP Codes for some of the data that they include in there, they are equivalent to the indicator warehouse. They draw from a variety of data sources, the analogous data sources that are available at the state level.
I hope that, as we move down the road here thinking about development, part of the conversation should be stimulating state and local developments. There are a bunch of counties, San Francisco is a leader and has always held up to the light. There is a variety of these initiatives at the county and community level, as well. Some of them have very interesting properties. Seattle allows actually folks to enter their own data in a framework, so they can choose data that is secondary data, as well as marry that to primary data. There are lots of different threads going on.
DR. ROSENTHAL: What might be really helpful, in terms of assessing where to allocate resources, in terms of bumping up depth or frequency outside of the privacy conversation, is just a basic kind of utilization or usage by the individual dataset. A kind of hit, download, maybe or something like that. If there are three of these that are accounting for 90 percent of the usage, that would be very informative in terms of being able to make an assessment.
DR. CARR: Jim, are you able to stay around?
DR. SONDIK: Can I say something? Let me just add something to Jim's really excellent description of this. There is another level here of data that I don't want the committee to lose sight of. Really, these tools focus on the aggregated, the indicators. A lot of this comes from raw stuff.
There are two kinds of raw stuff. This is my view, it is a technical term, raw stuff. There is the vital statistics, okay, which really is the listing of births and the listing of deaths, and it is as complete as it can be. Then, there is what we get from the surveys. The surveys is the really tough nut to crack here, because what you asked for is really hard to describe, when you asked for the diagram.
DR. ROSENTHAL: It is a specific technical entity relationship diagram. If you have a database, you do have an ERD somewhere.
DR. SONDIK: I don't know how to do that for a survey. I am sure if we sat down, we could do it in two minutes or so. The thing is that the survey, you get the data because you have used let's say a statistical technology beyond methodology really, to identify who you are going to bring in. Each one of those people, if you are surveying people, you can describe that individual in terms of what they represent, in terms of that diagram that you were getting at. They are a county, they are a block, they are a census, you know what I am getting at.
DR. ROSENTHAL: Yes, it is actually not a mini to mini. It could be as simple as saying, what states belong to a region, that is the type of thing I am talking about. If you are coming into this site for the first time, you don't know anything about the data. You see region up there, my first question is which states belong to a region. That, in terms of a diagram, is what I had expected to find as a developer. Say West Coast region includes.
DR. SONDIK: I am just saying that a lot of that thinking, when you look at an indicator, like the indicators in Healthy People, that thinking has already taken place, and that can be done. When I look at a survey, it is not so easy to do that. It is there, but it is many things. It depends on how I combine that data, that raw data, and how to combine the raw data, the 5000 individual, very, very long records, but there is only 5000 of them, in one year of HANES. There are many ways to combine. Do you agree with what I am saying? There are many ways. You can disagree if you want. There are many ways to combine that. The same goes true the 125,000 in HIS. That is a tremendous source of data. Actually, if we could get a few apps that start to use this, it will become clearer to the applications community as to what can be done. There is a challenge with this. I don't know of any apps at this point that go into any of the survey data.
DR. VAUGHAN: What would be your top three most important ones that you would want to have apps developed to?
DR. SONDIK: Well, I mean, there is the HIS, which is the core dataset for the department, a wide variety of information on everything from prevention to services that people are receiving. It is a nationally representative piece. That differs, in a sense, from what you get from CMS, because CMS says this is exactly what is going on, although there are surveys in there, too, that say what is going on. It is that survey thing that I think is really a challenge.
There is one other thing that I wanted to put in front of the committee, that is interesting. We did a briefing yesterday for Hill staffers. Somebody came up and said, I think this thing you got is really terrific, so it is very much on my mind. It is the tutorial that NHANES has. The tutorial is extensive, very extensive. It is not a two-minute deal, it is an extensive tutorial on how you get the data, how you analyze the data. It is actually award winning, and people can get credit for this.
Because it is so extensive, it is not as accessible as perhaps it could be. There is a possible application. It is not the usual app, what I would think of as the usual app. It is not just in front of you, per se, but the committee, because this is part of what we all do. We all being the people who produce this core data. We aren't as facile and as creative as we can be, if we can liberate this. The stuff I think we need to liberate is not only the data that has been massaged, and what I would call indicators, but what leads into that.
DR. CARR: I think it illustrates exactly why we have this meeting, because we have questions that we didn't know existed. He can use it, but you can't use it, what you need, he might need to make.
DR. ROSENTHAL: I think we might be speaking that kind of cross purpose. Just so you know, my Fulbright post doctoral was on qualitative data, so I have a reasonable approach to that. From a development point of view, it is kind of 80/20. For every database, there exists an entity relationship diagram. I absolutely guarantee it. That is cold, hard facts. You may debate about the nature of the relationships, but there is something behind that.
If you are wondering why there are no applications being developed, first thing to look at is market utility. Is any developer able to create anything valuable out of that, which is an interesting question. The second is, how easy is the access. The first thing a developer is going to look at is some form. It can be sifted, it can be only 20 percent of it, it can be 80 percent of it. Any basic metadata relationships, before I am even going to bother, because otherwise, I click on that and I don't go any further.
DR. CARR: I think this is exactly the kinds of stuff that we will begin to do. Let's hear from Allison.
DR. SONDIK: I want to give just a rejoinder to that. If you look at genetics, there are a lot of tools that have been produced that nobody around this table is going to use. Those tools are absolutely essential to the people who are doing genetics research.
In a sense, I have that kind of thing in mind, I think, when I talk about these core survey activities, whether they are what we do or what SAMHSA is doing or the many surveys that CDC has underway. I don't know how ready they are for the public.
DR. COHEN: I think they are. I disagree with you. I can see many uses, and this is a much longer conversation. If I am a woman and I just got pregnant, I am going to want to know what the C section rates are for a 35-year old woman with my particular risk profile. I would love to be able to go to an app that would combine the hospital discharge data and the birth data for my city or my community, and it would tell me, this place is good because, this place is bad because. All of those data exist in different places. I think the surveys can provide that kind of information, too.
DR. CARR: I want to make sure that we have time to get a scenario, to do just that. We have talked about that this morning, and want to hear what Josh has to say, too. To say, here is an issue. How would we populate this with the information, where would we go? Hold your thoughts. Allison, please.
Agenda Item: Overview of CMS Data Products and Services
MS. OELSCHLAEGER: I am Allison Oelschlaeger. I am from the Centers for Medicare and Medicaid Services, our new Office of Infomraiotn Products and Data Analysis that was launched, as I am sure many of you know, back at the Datapalooza in the spring, late spring, early summer.
It has been a whirlwind in the past couple of months, getting up to speed as the new office and really starting to think about the chart of our office is supporting both internal and external data users, CMS data users. Really, having a focus on data and the people who are using data out there in the world, and inside of CMS. There really hasn't been a focus at that level, until we announced this new office.
I am going to walk through a couple of the various data tools that we have out, starting with the Health Indicators Warehouse. I am actually going to stick to the web and show you some of the various things that we have on Healthdata.gov, look at the Blue Button Initiative, which is a cool thing that we are working on right now. Then, talk a little bit about how we are updating and improving our process for sharing actual claim level data with researchers, with states, that kind of thing.
Starting with the Health Indicators Warehouse, and for the people in the audience who aren't here, I am at HealthIndicators.gov. CMS has a specific kind of way that we present our data in the Health Indicators Warehouse that is a little bit different from how the other data is presented. We like to look at our data in specific report, so taking indicators that are related and having a report that shows all of them across a geographic area.
I am on the resources tab, and down at the bottom is the CMS indicators. We have our methodology paper that you can click here and kind of read about our various population and the things that we are doing to clean the data up. Then, we have our various reports. I am going to jump into the hospital and patient report, just to give you an example.
This is Alabama. One of the things that people are always interested in, and there will be more interesting data as we have trends in here, which will be loaded very soon, inpatient admissions per thousand beneficiaries. Alabama is obviously above the national average. When you click in and look at the HRRs, let's pick Tuscaloosa, it's even higher, 436.8 admissions.
What if we want to say, well, how does Tuscaloosa look compared to the areas around it? You can click through here and look at the data itself for just this indicator and jump to a map. Obviously, one of the things that you always have to think about when you are looking at individual indicators is the various interactions that it has with different populations. For example, maybe Alabama has a lot of dual-eligible beneficiaries who have higher resource needs than the general Medicare population. This is just one way of looking at indicators. It is not comparing across various sets.
Looking at the hospital referral regions, you will see down here that, here is Tuscaloosa, and it is surrounded by HRRs that also have higher inpatient admission rates per thousand beneficiaries. It helps you start thinking about targeting resources, or looking at causes for that in this area. We think that the Health Indicators Warehouse is a great tool, and we are really excited to see our trend data get up here, because it is even more information for people to start using, and putting into apps and that kind of thing.
I am going to jump over to Healthdata.gov. You can get to the Health Indicators Warehouse from Healthdata.gov. One of the interesting things that we're just starting to come out with is called Basic Standalone PUFs. I heard you guys talking a little bit earlier about kind of privacy and HIPPA, and thinking about how do we release clean level data, while worrying about privacy and making sure that we are not releasing beneficiary-identifiable information.
This is kind of CMS' first crack at doing that. These are public use files. They are available for download on the CMS website. What we have done is we have worked with a contractor to strip them of all identifiable information. It is still actual claims. It is not aggregated information. It takes things like age puts it into buckets. It reduces ICD9 codes to three digits instead of five. It is keeping hopefully a lot of the information that you need, while allowing you to download claims data, but also protecting beneficiary privacy.
If you click into these and you are not impressed, which was my initial kind of perspective, I think you have to remember that HIPPA has a lot of really strict rules around cleaning data, to get rid of identifiable information. We really have to be careful and make sure we are not releasing beneficiary information.
DR. COHEN: Allison, does this have the three digit ZIP or what geographic level?
MS. OELSCHLAEGER: I am actually not sure what the answer to that is, but if we click, I think it will tell us. There is some geographic information, I think. Each of the basic standalone claims have their own kind of rules around cleaning things. This one, it doesn't look like it has geography, but it does have the age category, sex, ICD9 code. Inpatient may have some of that information.
DR. FRANCIS: Could I just ask you what the de-identification methodology is that you are using? Is it the Safe Harbor?
MS. OELSCHLAEGER: It is not the Safe Harbor. We are actually going through statistical review with our contractor. They are both masking, kind of creating groups and categories to mask things, and then also making sure that there aren't any cells where you could aggregate and have it count if less than 11 beneficiaries or providers.
DR. FRANCIS: Who is the contractor?
MS. OELSCHLAEGER: NORC, yes. It looks like we are not doing geography right now.
MR. SCANLON: Is that the national file, all of the national level claims? There's no geography.
DR. FRANCIS: You can't get it by state or by region?
MS. OELSCHAEGER: Right, it is a 5 percent file. The other thing that I wanted to show you on Healthdata.gov is our Medicare and Medicaid statistical supplement. That is in the Medicare box down at the bottom, and it is the fourth one down here. This is something that I don't think all that many people know about, but is a really great resource if you are looking for basic statistical information on the Medicare/Medicaid program. It has things like enrollment by state, enrollment by eligibility type, all in individual Excel tables that you can go in and look at.
One of the things that our office is trying to do is improve the way that we start sharing data. Right now, this is just Excel tables that you have to download individually, but more and more, we are trying to think about how do we move into the iTools or other ways to share this data that gives people more access and makes it easier to use things.
Then, the final thing I wanted to show you on Healthdata.gov is the compare tools. Medicare has a rate set of compare tools. Probably the one that is the best known is Hospital Compare, and that has been around for a while. Now, we have Nursing Home Compare, Dialysis Compare, and that data actually sits on Medicaredata.gov. Let's go over here.
DR. VAUGHAN: Allison, what is the recent year for this?
MS. OELSCHLAEGER: For compare data? I actually don't know the answer to that. It might be '10, but I can check and get back to you. Hospital Compare, Nursing Home Compare, Home Health Compare, and these are tools that allow people to come in and actually look at providers and see how they are doing, and compare across. In their geographic area, for example, what are the different providers that they could use and what are the comparisons on various indicators of quality. Medicare.gov has a setup that allows you to automatically do things like filter and visualize the data yourself.
I am going to jump over to MyMedicare.gov. Another initiative that we are working on is the Blue Button Initiative. This is a way to share claims data with the beneficiaries. One of the most recent changes that we have made to the Blue Button is to expand the amount of data available to beneficiaries. They used to only be able to get one year of A&B claims data. Now, they are getting three years of AB data and one year of part D data, which is a great addition.
Scrolling down, it is right here on the main page. You can click Blue Button, download my data, and it opens up a website that allows you to read about the Blue Button program. You can actually go ahead and download your data. For example, at Datapalooza, we saw a developer who is working with Blue Button data and helping patients start to use their data in more interactive ways on their iPhone or Android or whatever, and actually share the data with their providers, and then have the providers share the data back with them. There is a lot of tools, and this is not only a Medicare program, but also a VA program, which is where it started, and also some of the FHB plans are starting to do this, as well.
Then, the final thing that I wanted to talk about with you guys was the Research Data Assistance Center. This is our external facing group to help people who are interested in actually getting claims data, or Medicare current beneficiary survey data, identifiable data. They are going to sign a DUA, they are going to promise us that they are not going to share the data or sell it, or use it in incorrect ways.
ResDAC just designed their website. It used to be kind of old format. It looked like you were going into an university webpage from the early 2000s. We are really excited about this new website, and all of the kind of improvements that they have made in communicating with researchers and communicating with states, and making sure states can come in and get Medicare data and can share Medicare data within the various agencies in the state.
Going back to your earlier point on having a program for people to go look at quality of providers, in the winter, we announced the Medicare Qualified Entity Program. That is a program, qualified entities come in through ResDAC to request the data. It is a program where you can get Medicare data specifically for public provider performance reporting. We are really excited about that opportunity. It gives us a new way to share our data with people who are going to use it for good purposes. They have to combine it with other claims data, so you are going to have one report that kind of covers the provider's practice, Medicare, Medicaid, private plans. That is an exciting thing, too.
I want to give you guys a chance to ask me any questions.
DR. FRANCIS: I have a question about your data use agreements. Actually, first of all, I would love for us to be able to get a copy of it. Secondly, I am curious about whether you do anything with respect to following up on whether people comply, any spot-checking, anything of that sort. And what the penalty would be if somebody broke one.
MS. OELSCHLAEGER: That is a great question. Our data use agreement is available publically, but I can also make sure that the working group gets a copy of it. There are penalties in the DUA for violations of the agreement with CMS. In a lot of cases, I think we have found that most of the people who violate the DUA don't mean to. They are not selling the data or whatever, so we try to work with people to make sure that we are not penalizing them for doing something that they are not necessarily at fault for.
In terms of follow-up, I think that is something that CMS is kind of still trying to figure out the best way to make sure that we are not. It takes a lot of resources to follow up and make sure that you are tracking people. We, so far, have only shared our data with kind of trusted academic researchers.
As we move towards more data sharing, one of the things that we are considering is a data enclave model, so giving people virtual access to Medicare data in an enclave. That gives us a little more control over the data, making sure that they are only using it for purposes that we kind of think are valid, and making sure that they are not running off and doing something else with it. I think that is a good solution, moving forward, and we have already started piloting an enclave with 200 users.
MR. SCANLON: Allison, on the follow-up question, ASPE, we actually did a joint project with CMS privacy group, and designed a program for sample audit follow-up. It is just a pilot basis to see if it works. They might have actually tried that, Allison, I am not sure. It was basically not just complaint-driven, but to actually follow-up proactively on a sample of some of the data holders.
As Allison said, it is usually something like the researcher added two graduate students to the access list and didn't tell us. It is usually not more serious that than, but it would have come to our attention.
MS. OELSCHLAEGER: If you are really interested in how we are approaching privacy, there is a whole privacy group at CMS that knows a lot more about this than I do. It would be great to have them come in and talk to you.
DR. MAYS: Is there a tutorial or a place that you can go to? What I am thinking about is, in terms of other academics and students, et cetera, getting access, is there some place you go and it would give you a kind of a sense of all of the datasets, or would I have to kind of know the different things?
MS. OELSCHLAEGER: In term of beneficiary level information, if you want to do your own research, or in terms of kind of the various data that information products that CMS makes available?
DR. MAYS: Yes.
MS. OELSCHLAEGER: There isn't. We are announcing very, very soon, there will be a data navigator function on the CMS website. It is pretty much ready to roll out, but we are just finishing up a couple of things. Our website is notoriously difficult to use and to find things on, so the data navigator will be a great tool to help you find the specific, if you are looking for Medicare inpatient information.
MR. SCANLON: ResDAC actually has got a little ways towards that. It shows you all of the datasets they have.
MR. CROWLEY: Do you have any mechanisms in place to understand how the users that are coming to you are making use of the data, or cataloging their needs and wants from the data?
MS. OELSCHLAEGER: I don't know the answer to that question. When you guys asked Jim, I was thinking we need to figure it out if we don't have it. The CMS website is not something that our office manages, but the office communications that does manage it probably does collect that information. If they don't, they may.
MS. QUEEN: There is a CMS privacy board, and so when things are requested, restricted data is requested, there is a whole protocol that gets submitted, that you know exactly how the data are going to be used by whom and for what purpose.
MS. OELSCHLAEGER: When researchers come and request underlying claims data, we do collect that information. The statistical supplement, how many people are going on there and downloading tables on Medicare enrollment, I don't know if we actually are collecting that kind of information.
MR. CROWLEY: As a researcher, it is useful to understand how other people are making use of certain datasets, from techniques to methodologies, to the extent that they are willing to share.
MS. OELSCHLAEGER: ResDAC is doing a great job of starting to think about ways to share that information. They are going to soon have a page up that will allow researchers to come in and say, here is what I am doing with the data. Here is where I have published, and then that will be available to other researchers, as they come in and think about, okay, here is what other people have been doing with this data. What can I do to add to that?
DR. KAUSHAL: What is the time lag between updating the data? For example, claims data, as an example.
MS. OELSCHLAEGER: Meaning for access to researchers or public products, everything?
DR. KAUSHAL: Yes, I'm sure there is a range.
MS. OELSCHLAEGER: Yes, there is a range. Historically, CMS has been almost at like a two-year lag, in terms of getting data out publically. Right now, we are at probably a year lag for public data, so for kind of stuff that we are getting out and sharing, maybe a little bit more than that. As Jim said, we are updating the health indicators warehouse with 2011 data now.
Six months to a year is when we kind of feel like we can start to get data out publically. For researchers and for qualified entities, we are starting to move more towards a six-month lag. Between five and six months is when we really see the data starting to settle down, and we are doing more and more work, trying to figure out when can we actually share data, and what is the earliest point that we can get data out there.
DR. KAUSHAL: The aspiration is to cut that six months down to even less?
MS. OELSHLAEGER: It is, except that we are really relying on providers submitting claims. When they don't submit the claims and the data is not complete, we really can't do anything about that. I guess the other option is to tell people the data is not complete, and it is up to you to figure out.
DR. CARR: That is what we are doing with the Pioneer program, getting rolling data.
MS. OELSCHLAEGER: Yes, as we are starting to communicate with providers, we are giving them even more up to date data.
MR. DAVENHALL: I want to make a comment. I wish Walter were here, because this is a standards issue. Why I want to point this out to you is that, one of the other problems we have to address is there is no standards in how people are setting up these websites. We saw the National Center, and they have this look and feel. If you want people to really start to use your data, you have to start to worry about being consistent across the board, as to what these things look like. Otherwise, people go to these sites, spend most of their time trying to figure out where this stuff is.
If you look at this site, this is, in my personal opinion, the richest, best site of all of the sites that we have been talking about here today. I would ask you to go to that one and play with it, and download that file she has out there, the provider number. Now, this is one of the most difficult crosswalk files to find in Medicare. Right, Allison?
This has every hospital and provider, and has their Medicare provider number. If you have that Medicare provider number, you can link that to a whole bunch of interesting statistics from intensity rates in hospitals to DRG adjustment rates and so forth. You have got to have this crosswalk file available. You can download that.
If you look across this tab, right here, you can't read it from where you are here. It is the most fully functioning set of tabs for anybody to get ahold of data out of CMS. I am just proposing that, as we go down this road, we think about using something like this as a model. It was immediately obvious what you are going to be able to get from that site, where the other places, it was always, can I download from this site, can I export, can I share, and that kind of thing.
The other thing that I wanted to mention while Jim is still here, tell Ed there are people out there who are using his data in the way he said he has never seen it. We need to find a way to bring him up to speed on that. I would say 50 percent of the hospitals in this country are using that survey data in a way that he would be shocked.
MR. SCANLON: Can I ask Allison, so this is institutional provider? It is not individual professional provider?
MS. OELSCHLAEGER: We don't make that available, although the qualified entity program is moving in that direction. As we start to announce QEs, that is pretty much what they will be doing.
MR. DAVENHALL: There is a file called the hospital service area file, by the way, which now you put data out in six months. That file has 2011 data in it, of every dollar that Medicare paid into a ZIP Code. It tells you how many days, how many cases and so forth. It provides a provider ID. It is totally worthless to you, unless you have this file, to tell you who that provider is.
I would say some of these things, as we think about, as we do our work, how do we make these crosswalk files? Could there be another file there, when you go to that other hospital service area file, that says, oh, just link here and you will get this crosswalk file, kind of thing. I really want to compliment you on this site. I would say that if more of the sites had that kind of look and feel, we would all get to it easier.
DR. ROSENTHAL: If you do that taxonomy work or see the entity relationship diagram, that stuff will pop out. You will say, oh, we don't have that, and that will be your roadmap for your metadata development, fi you will, to say, here is where we need to build something, as Bill said.
DR. CARR: Josh, did you want to come forward and go through?
MS. GREENBERG: I know Allison mentioned about the data use agreements, et cetera. You were pretty much talking about public use data tapes. I don't know if everyone on the group knows about our research data center, where we make more access to more restricted data. I just thought maybe you could just say a few words about that, and then maybe, in a future meeting, they might want to hear more about that.
MR. CRAVER: I mentioned our RDC, our Research Data Center. It might not be a bad idea to have Peter Meyer give a quick tour of the concept. The Research Data Center is a facility really that is under lock and key, and restricted access part of NCHS as a physical space that researchers can submit proposals to, and if accepted, come to our place and have the access to restricted datasets and datasets that are non-public, and do their research on it.
There are a couple of caveats to that. One, it costs money, because you are using our facility and our staff to do that. We have to set up the files for you, so that you can have access to it, and we also do disclosure review of your resulting files, so that we are not releasing into the wild something that legislation should. There are non-trivial reasons why we care about that. One of them is if we do disclose data, we, as individuals, are on the hook for hundreds of thousands of dollars, and many years of our lives in jail.
DR. COHEN: It is important to know that, not only NCHS data, but we actually are going to be using the RDC as a data enclave, to store data from another part of CDC that we are linking with state data. There is no way that we could get it out of CDC, because of the agreement that CDC made when they actually collected the data. You don't have to go to lovely downtown Hyattsville to use this. Remote access is a key for using the data at the RDC. It is a really powerful option that really hasn't been explored fully for using confidential data.
MS. QUEEN: Will the CMS data enclave be like that, be like an RDC?
DR. COHEN: Yes. The census uses, as well, a portion of data enclaves.
MR. CRAVER: At NCHS, we have ongoing and developing relationships, new relationships with census. NCHS, RDC now installed in Atlanta, there were recent discussions at a university in a laboring state to install a new RDC that would have census data, as well as NCHS data. It might be the time for us to discuss with CMS.
DR. VAUGHAN: Jim, is it now possible to align CDRC data with RDC, or is that a special super security? Is it possible to unify that under one secure roof at this juncture?
MR. CRAVER: I think that that is happening. I think that the efforts that need to take place are in process.
DR. CARR: We have an hour and four minutes remaining. This has been very, very helpful, very rich. I would like to hear from Josh, and then hopefully have 45 minutes or so to talk about what would be the next thing that we might want to do, to begin kind of getting a feel for what is available, how it works, what the issues are, et cetera.
DR. ROSENTHAL: I had from a previous meeting and this meeting, I just decided to put together some slides instead of put it in the Wiki. I had roughly speaking, and these were some suggestions and conversations I had with Todd way back when.
I had about seven very basic recommendations. Some of them are crazy and challenging, some of them are kind of common sense. I know Greg and company and Allison and John are already working on some of them. The specific recommendations, where to start, would be taxonomy, communal learning center, baking in business or market value into the challenges and contests, files accessible, the NORC IMPACT stuff you are talking about, I would also add synthetic files, completely synthetic for priming and loading systems, as well as just getting rid. I know you might already be doing that. Just leave it out there for the whole committee, so everyone is aware of it.
Data browsers, which are very different, Google Public Data, TABLO data, where you don't have data available to people. You just allow them to do analysis on the fly. This is common in the web world. Some of your data was actually used by TABLO and ReadWriteWeb at a contest, half a million people used the thing. Someone won, a young girl, for diabetes, comorbidity. It was fantastic, your data was being used. It was in this browser, they could instantly ask questions.
Then, partnerships and product development, we got into some of that. That will take me into basic web utilization. This is Google analytics stuff, so you get a sense of who is doing what. Opt-in, which is completely crazy, but consider it like green button for people who want to have their data shared. Don't underestimate that, as everyone knows from Blue Button.
What is taxonomy, just really quickly so everyone knows what we are talking about. It is not just metadata or talking about it or putting out sentences about it. It is showing the relationships between the data. Before you do anything else, before we get into it, you have to have a taxonomy, and you have it one way or the other. It is there. I can reference internal or external conversations.
Let me give you an example. I am looking at this file from CMS. I download the thing and this is what I see. I am like, all right. I am a developer with no health care experience, now what am I going to do. What are these things?
Well, I can go and maybe I can find it in a metadata catalog, but that doesn't tell me anything I need to know to develop an application. I have to sift, bit through bit, and be able to reconstruct the relationships. It turns out that plan belongs to contract. I mean, you get into kind of mini to mini, it belongs to organization, belongs to parent organizations.
Taxonomy defines business entities and the relationships between them. Parent orgs have orgs which have contracts which cover plans. Then, there are attributes, right? Contract number, legal entity, this little cell on the spreadsheet, what does that belong to? That will kill a developer coming in, who has no familiar with it. They expect to see that sort of thing.
Anywhere else outside of health care, you see it. You see it in government geolocation, you see in weather. All of the great success stories we have talked about have this. If we are wondering why it hadn't picked up, if we are only hitting 1000 of user downloads, this is really, really important stuff. There is sort of an expectation in it. It doesn't have to be. I understand the privacy thing. It can literally be what states belong to a region. It can be that level, same thing applies, 80/20.
If you create all of this, you have CMS doing their data products and tools, Niles' office, you have researchers doing their things. You have people in the commercial space, who will build applications, if they can create market value, myself being one of them.
Inside your extract system, call it nonclave, you need a public learning center, I will get to you. The center of all of that is taxonomy, the business relationships of the entities. If you want anyone to build something cool up here, it is very, very difficult to ask them to kind of knit by hand all of those relationships. I just gave you one example of a file. Thousands of them, to be able to pull together a reasonable application. That is taxonomy.
Learning center, I will let you read on your own time. I want to be able to see what other developers are doing, and actually how some access, Bryan talked out speaking with the data experts. We could do that at little roundtables and committees, but actually having an interactive community to do that would be really, really helpful.
You know what would be really helpful? Sharing the web analytics, not only so you have them, but so that when I walk into it and I see 500 files, I say, what is the most important one? What file is everyone going to, right? I know that Bill is a super user because he has 5000 posts and five stars and five recommendations. When he says that particular crosswalk file is the most important thing, I am going to listen to him. That sort of information, doing that in a scalable way.
I might suggest, and even Bryan mentioned it, this doesn't have to be kind of big budget, cost-built internally. You are hosting various challenges and spending money for this or that. Point it towards the core infrastructure, and you will get the developers themselves actually doing this. Learning center, blah blah blah, all of your videos, blah blah blah, okay, here is what it might look like. I know they are working on this. You have different things in there, fine.
For the challenges, you track an internal database, or you look at crunch base, in the health startups. This is some of my work and I have had a couple of successful ones myself. They fail at an astronomical rate, far outside peer play technology. Why is that? Legacy guys don't want to retool, fine. The young MIT and Harvard kids come in, and they can't navigate perverse incentives, they can't figure out the market layout. They don't know the business case of the data they are looking at or how to use it. They built WheatCracker 2000. If you guys want to steal this app, you can. How fat am I today, a pig, a hippo or a Jabba the Hut, right? This is what they come up and they try to sell it at DTC and it doesn't work.
If you are building a challenge and you are saying, I am going to spend money and reward someone for building an application, one of the criteria for building that application or product should be having viable market value. I am not talking about a whole business case. Todd referenced the HDI analytics and data session I put on at HDI, and where I had a little tiny Mad Lib template, where people had to fill it out, and very well reviewed. That was just literally just forcing them to say, what is the market, what is the product, why would they buy this, what are the challenges, what are the opportunities.
Doing a little bit of kind of business thinking around the data, I would humbly submit would astronomically increase the success rate in these challenges. By the way, if you have a learning center, you can put all that information up there so I can see. I see that he says this and it is working well for him. I say, oh, that is a really interesting use of that data. I never thought about using it that way. Then, we can build on one another, and that is sort of what you try.
DR. KAUSHAL: What do you see as our role in teaching that business case?
DR. ROSENTHAL: I am going to humbly refrain from answering that question now. What I am going to say is that I think, however we do it, if you want to crowd source it, if you want to experts, personally, I think multiple perspectives, I would love to have James, I would love to have the West Wireless fellow talking about some of that. I would love to have some of the challenge winners talking about that.
By the way, if you are spending dollars internally to kind of track, I might look at the companies that have successfully done this and had exits, which are few and far between. You could have a multiplicity of perspectives and uses.
MR. CROWLEY: Essentially, you can wait the scoring of the challenge to have a model as one of those factors. We do this when we run challenges at the business school. We will bring some engineering students, get the business school students, public health students, and it is really helpful. Will you bring in either a resident or the business expert residents, sort of throw them in.
DR. ROSENTHAL: If you capture that and keep it as part of it, and now I go to the public website and I cannot just see the data, I can see the relationships of the data and this taxonomy, business relationships. I can see what people have submitted and how they are trying to use the data for different business issues. That is how you develop and create an internet time.
DR. KAUSHAL: There is a supply side of data and then there is the demand side. Our role in this working group, my assumption was a little bit more on the supply side, but would love clarification. I think that you need both.
DR. ROSENTHAL: I am just throwing out it would be good to get both. Basically, before we even start saying what is more important, where should we focus, there is a demand issue. I might humbly suggest that a little tiny form, in terms of demand and what you would like, with some basic categorization. The stuff you consider, like the iterative quick form, as well as just the basic demand issue, who is using what. Let me just get through and then we can go on anyway, because I have a couple of more.
If I actually win this thing, I am talking about it, and by the way, this has worked really well in other instances. You are doing public data, blah blah blah, perhaps offer some synthetic or fully synthetic. That is really nice for reasons you will get into. A lot of the big folks, if I want to build an application, which I did, and prime the system, I need something. I don't need security clearance, I just need fake, made up data in your structures. You will hear the big analytic vendors and big payers say that sort of stuff. You have already created it.
If you did that in a very specific way, you might even be able to kind of walk around a good deal of the demand for privacy. Pure synthetic files, if it has a specific type of meaning intact. Here is someone at Caltech. She isn't in health care whatsoever, no affiliation with what you are hearing earlier. They use the stuff for privacy, for credit cards, pure synthetic creation that retains a specific analytic usage. It is done under industries that are out there.
Push the ERD, here is the data browser. If I go to Google Public Data, I will actually be able to see all of the wonderful data and I will be able to move things around, and click and ask questions. This gets tremendous usage. If you want to engage the broader community, who is not even data savvy, but the tech business students who might want to get in and ask, what might I want to solve in the health care, starting from the business side, before I hit the data. This is tremendously valuable. These are data explorers, free, Google has one, TABLO has a public one. I mentioned they did a ReadWriteWeb one, there is a Google one.
It got like half a million hits, half a million uses in their contest, right? The contest was they put up a little bit of data, some of your stuff publically available, about a half a million hits. A young girl wins it with comorbidity of diabetes. Why does it generate that usage? She doesn't have to understand the data structures. One of the people actually put it together, and then it allows everyone to play and ask the questions. No data leaves the environment. It is all de-identification, aggregate. That is very good partner usage, basic web analytics, demand and supply is obviously a question, but just somewhere to start with.
Partner publicized monitor, and then you can actually allocate resources and build based on the most popular stuff. I did that with Todd, so it's kind of a joke. You do a photo gallery of Todd and it is the most popular thing.
Finally, Opt-In. Blue Button was crazy, no one will ever use Blue Button, right? You guys remember all of the talk before it went out, 10,000 uses or something like that. What is it, 10 million now? How about just a crazy idea of allowing individuals, like myself, there might be some others, who want to share their data for specific usage. I want to Opt-In and say, please use my data for research. Call it a Green Button and see what happens.
Obviously, you have to work out all of that, but this is another kind of creative way to attempt to get around the privacy. If I say I want the Opt-In for limited usage in a crafted way, if you get even a very small percentage, and if you get something like Blue Button, you might be surprised. All of a sudden, you are creating another very rich source of data. That is it, so thank you very much for the few minutes.
DR. CARR: Jim, did you want to say anything?
MR. SCANLON: No. I think we are at the point now where we just do some brainstorming here and what do you think would be a reasonable next step.
DR. VAUGHAN: One thing I would also suggest, too, is there are use cases, and looking into some of the things we just talked about over the last couple of days. The products that you are looking for don't just have to do with building private businesses, that part of the audience and part of the customer base is public health departments, non-profits. Their use case and their outcomes and their products and their needs and engagement issues are going to be different from a developer oftentimes. They may be far more data literate than somebody who is trying to approach it from the point of view of, I don't have to know any data.
I think that also we shouldn't lose sight of what is a very rich public health heritage, and that people have been donating their data for many, many decades, in the Washington County study, the Alameda County study, in framing in the national nurses study. Much of that is with privacy protection, but has offered a very, very rich template upon which we have learned a great deal. Perhaps move those forward in ways that make it more accessible, I think, is not going back to reinvent the wheel, but use what we know works really, really well.
MR. SCANLON: I think in terms of developers, the motive is not the interest. It is more, if there are public health person apps, that is fine. If they are for profit, whatever they are, I think we have experience at reaching some of those audiences, and we have formal ways audiences. This community, we don't.
We don't have much experience and obviously we need your ideas about how to get those two communities together. We have the auspices of a federal advisory committee. Bring people in, whatever they would be able to say publically, we could do that. If there are some questions we should pursue, in terms of environmental scan a little more. I think you are right. I think what we will do is look at the usage statistics. It might be embarrassing, because you have to promote.
DR. VAUGHAN: There might be an opportunity to say, well, given what we know, we have use statistics now. What would be helpful and useful moving forward, if we found those that weren't quite what we might need.
MR. SCANLON: If you could give us ideas about the metadata, here we are, we just renewed and revised the metadata for the Healthdata.gov. If we are missing a couple of more items, obviously you can't have infinite metadata. No one would ever put up that set. If there is a tendency toward a standard or at least a core that would make it available, not only for what we have to do in a government agency, but what would make it easier for developers, that would be very helpful.
DR. VAUGHAN: I guess to not discount that there the potential community of developers is far broader than you might imagine. To try to think out of the box, so that we are being as inclusive as possible, because I hate to think of these very rich domain experts, who are in struggling health departments all around the country. What are those people doing, when they really should be tapped and brought to the table.
MR. SCANLON: I would think all 3000 counties shouldn't have to do independently a decent set of community health indicators or population analysis. That is more than just the data. That is what we do in the U.S., everybody does it, that is an obvious. Everyone shouldn't have to do it. If someone makes money doing it, too. Otherwise, you have to be an aficionado. You literally have to know who does this, how do I find this, how do I pull together. There are aficionado.
DR. ROSENTHAL: Even if you are an aficionado, I will speak for decades worth of Dartmouth atlas, I still as an aficionado say, where is the ERD. It is not mutually exclusive, it benefits everyone.
MR. SCANLON: That raises more questions than it answers.
DR. ROSENTHAL: Even the super savvy aficionados, basic kind of community sharing, if government doesn't want to share, if you want to say actually, I don't want to share this, or there are security reasons, or we don't have an official taxonomy, which we don't, there are multiple ones floating around. If you take kind of the community development perspective, I share mine with other folks in the community, and they share theirs with me. It would be really nice if we could post that somewhere, where other people can share that, as well.
DR. COHEN: I think the sharing issue, we can certainly figure out how to get past that. I don't see that as an impediment to doing what we want to do. I see the impediment from my point of view is, we are the data holders, we understand this really incredible set of very complex and data that we think is useful for certain things. We don't know how it resonates to the real world, because we are these geeks who focus our entire lives on making the distinction between 37 weeks and 38 weeks gestation.
Not only do I need to learn to speak your language about the kinds of stuff that you need to develop the apps, I need another perspective to ground my distorted reality about how these data can be used and applied in ways that the folks that are actually developing these applications know resonate with communities who might want to use this. I am asking just as much for guidance in that area.
I am having trouble thinking, I could name a million business applications that I think would be cool. I don't know whether anybody else would care. That is the kind of feedback that would help me figure out how we can best get you the information that you need to do your development. That is where I am coming from.
MR. CROWLEY: Basically, that question that you just asked, Bruce, there is not enough staff or time in the world to curate and catalog all of the potential uses of this data and find out people's needs. As was discussed, and has been discussed in different ways, by virtue of a learning community with certain social features, to allow people to have that conversation, for you to also engage in that conversation with them, as they had their questions, you bring their expertise and have it in a community-mediated way that is accessible to others, then that creates those answers to those questions.
DR. COHEN: That is a great idea. Just say, here is all the stuff we have. What can you do with it and what would be valuable for you, or people like you, and begin that conversation. Perhaps, priorities would emerge.
I think we have some basic usage statistics we have for our web-based query. We see what reports people like to generate and stuff like that. It is the people we don't reach. We have a cult community of data users, because our data has never been free enough for people to get access to it. It is the people who don't use the data who we want to be using the data, not the usual suspects.
I think learning community is the way to engage folks. Show them what we have, in a way that they can understand it, and get the feedback so you can develop the tools that they can use to meet their needs.
MR. CROWLEY: You might want to consider incentivizing in some way. There is a community willingness to learn an approach, but maybe run a couple of pilots. Some of the challenges that have already been used for leveraging those mechanisms, to build that into the community. There is some additional quip or quo for participating.
DR. CARR: Okay. Now, it is time to land the meeting, at least for our next thing. I am trying to keep up with you all here. We talked a bit, there are clearly infinite numbers of ways to approach it. There is a need for us to articulate what is it, what are we doing to do. Is it that we are going to talk about, as you said, the uses of the data, the usability of the data, the applications. I think all are valid, but we have got to figure out one.
I will just bring you back to what we talked about today. Because we are linked into NCVHS, we have an interest in getting more expert in what is available. One of our themes for this year is thinking more about our communities, the data available to them, what they do with it. It brought us back to this slide that came out as part of the NCVHS Shaping Health Statistics 10 years ago.
As we look at it today, there is a way we could make it come alive, and even test the validity of it, with the data that is out there. At the center is the population's health, disease, functional status, well-being, the incidence and the distribution. Beyond that, our community attributes, so some of these we could get. The biological characteristics, community age distribution, gender, genetic makeup, health services, number of personnel that are available, cost and financing. These are things that I think that we have been seeing today, that are out there, and population-based health programs and so on.
Then, beyond that, context, natural environment, and this goes beyond HHS, but air quality, water climate, weather, cultural context, political context, and place and time. Bruce, you may want to speak to this a little bit to say, would this be something that we would land on, to say if we use this to kind of walk through, do we have this kind of data. Does it help us, what are the complications that we encountered that we were not anticipating.
I just use this as something. I know it highlights Massachusetts because that is where I am from. This is something HRQ has put together, and it combines a whole host of factors, and comes back and tells you what every state looks like. I just want to point out New York, as opposed to Massachusetts. It is just something I happened to be looking into this week. This would tie in very well with our ongoing focus on our community. This is Bruce's idea.
DR. COHEN: This is the space where I want to be, but again, I don't know where other people want to be and how they see what we can provide. I am very interested in pursuing this from the national committee's point of view. I think this can add value to us understanding community, the health of communities and how people relate to that by essentially breaking down the traditional silos of defining public health very narrowly. I think this is an incredibly important space for the quality of life.
Listening to what Josh had to say and some of the folks today, I don't know whether we can develop applications from this that people will want to buy, if that is the bottom line. Again, I am very ambivalent. I really want to be here and explore the possibility, because I think we can add value from my perspective. Again, I don't know what the folks who once we liberate our data independently, what they are going to want to do with it. I have these two competing notions in my mind.
DR. CARR: Datapalooza and cool apps are sort of the enticements, but do we want to take it to a level of a little more sophistication that is beyond a cool app, but it is a relevant, sophisticated important question. I am not saying this is the only one, but I really like Bruce's idea that it ties in with the work of the committee. You have an audience, and it would take us through our paces if we did Healthy People 2020 and put that in there. I don't know if it is simplistic to think about rolling it up like that, that would be a really cool app.
Ed, you have been saying everybody is looking for the one number, right, that tells you about a community. I think also, as we look at this, we don't know what of these things, in context or community, drive health. We have been talking about it for 10 years, for a decade, but no one has ever put it all together to see if we could get one number.
DR. FRANCIS: Mostly what I have heard addressed is that the current data that are up there could be more user friendly in many important ways. What I haven't heard is, and this goes to what you were saying at the end, and it also goes to the choice question that Heritage put out, really an underlying problem is that the datasets that are currently out there just aren't rich enough, or enough interesting enough.
What do we then start to run up against with respect to stewardship questions, protection questions, and so on, if we are going to go richer. I don't know, and I think that is partly my role here. I just want to tell you that there is some work that the committee is doing, if you weren't here earlier, about data stewardship for uses of community health data. This group is going to get a copy of what we are looking at, before we do anything with it.
DR. CARR: I think of all the array of things that we have seen, there are a huge number of things that do not touch into that sensitivity, and when juxtaposed and integrated harmonize, can tell a very powerful story. I think that we have got to walk before we run. If we jump into de-identification, who has the right to know what, we are really going to have the same experience of others.
DR. GREEN: I would like to make an observation and nominate something for one of your PowerPoints. I know there are going to be next steps or goals or whatever you are going to do up there. I think I am safe doing this in behalf of the NCVHS committee.
First observation is, I have been back there listening to you guys all afternoon. So far, I have not heard anything that cannot fit comfortably into the three themes that the NCVHS is going to guide its work going forward. If you disagree with that, I really want to hear that dissent and understand what that dissent is about. This looks very promising, from my perspective.
The thing I want to put somewhere on your list, if you will let me, goes like this. Regardless of what type of sharing we have, when we stay focused on helping the people in the United States have better, longer lives, and to get the health care they deserve and need, we keep re-exposing, over and over again, that this nation is missing an infrastructure. It happened again this afternoon.
Infrastructure probably means different things in everyone's brain, sitting around the table. I am using that word to put a spaceholder out there for systematic proper use of new knowledge and new technology. We have these infrastructures for other human enterprises, but we do not have the infrastructure in place to either turn the health care delivery system into a learning system, or to turn communities into learning systems. What NCVHS has been seeing now for a decade is a silent cry for that infrastructure.
A minor, but I think important, example for NCVHS, I usually just try to beat this to death. I have annoyed Ed at least three different times about the workforce. We heard in our hearings that the analytic capacity of this nation's public health system is weak. In some places, it is not weak, it is absent.
We have heard that we don't have a fellowship program some place, that is preparing people to do the sorts of work that I hear you guys wanting to do here. Then, share it, disseminate it, teach it, augment it, scale it. Where is that national infrastructure that prepares the workforce to work within a new infrastructure, to just totally change health care and health status in this country?
I have passion about this now for several reasons, but the main one is, I will state this negatively, there is a risk that we are going to get a lot of cool stuff done and it isn't going to matter, because we didn't bother to think what it was going to take to move data into information into stories that change the world. I want to get that on our list somewhere.
DR. CARR: I am going to push back a little bit, because I think we can either come at it from here down to here, or when I look around the room and I look at the expertise, I am thinking if we do one thing, we will learn many things. I am wondering if try to land on walking through one thing, it doesn't have to be a long time thing. Let's see how it goes and that will obviously inform.
I would agree with you 100 percent about the workforce. I have been saying it as long as you were, even longer. Maybe it is too hard, so maybe this is the interface. Maybe we create information that is so easily accessible, that you don't need a PhD in these various things.
DR. GREEN: I don't hear that as a push back. I hear that as helping us with our question about how are we going to adjudicate and coordinate our work. If this group could get down and dirty, and while we are thinking about changing the world, if you could just change something. If you can get a couple of apps that just knock your socks off, and finally you say, now, I get it. I think the full committee is well-positioned to grab that and then to provide advice at a broader level, to do broader policy work that helps get that done.
DR. CARR: Which gets back to Bruce's idea, which is if we landed on this, the long-standing NCVHS kind of vision, and tried to say, okay, now we have made all this data available, let's fill in the blanks, what is it telling us. If we take the healthiest community and find out it actually doesn't matter at all what the streets and the roads and the air quality is. And take the worst community and we see, oh, actually, they have the best of everything. I don't know what the answer is. I don't know if that is a simplistic way.
DR. ROSENTHAL: Real quickly to interject, just pick something. Do whatever you are comfortable with. Do what you know to start with. You are not going to find something else you don't know.
DR. VAUGHAN: I would say maybe a lot of those questions are answered with GIS. What happens where and why, and how is it different. I would also say that, for all of the infrastructure we lack, we also have a lot of infrastructure to work with, that we are not using. It is not so much that that is legacy or old school, and this is bright, shiny and new, and a killer app. It is what is the best practices of both and bring those together, and making it possible for those to come together.
Agenda Item: HHS Approaches to Dissemination
MR. SCANLON: A couple of specific things I would like to get out of the group, not today. Number one, you all have ideas about, I wouldn't call it principles or guidelines, but something like that off the top of your head, about how examples of what you heard today, which HHS could do better. Very practical or theoretical, but again, that we could begin to. You made some very practical suggestions today. This is to improve what we are doing already. Maybe what we could do is to ask people to think about them and we will start getting it on our worksite.
Secondly, I see a lot of student level applications, I will call them student level, and a lot of enthusiasm. I am an optimistic. I think generation three, those will turn into serious, not that there aren't serious ones now. Again, I don't want to dampen the enthusiasm, but if you have ideas about how you get to that stage, how we could get there a little quicker in the health area, that would be helpful to us, including the idea of down the road demos or incentives or challenges. Again, I don't want to dampen enthusiasm. We want to be careful of how we use your time. I think we are looking at sort of beyond that.
Then, third, I think do we want to pick, and we are not ready to do this, an area. An area where it seems to want community level data, and there are a lot of examples of that available now. I am guessing that many of these tools are ignored. You can assess the use of the various tools now. We could focus on this at a hearing, what differentiates use and application from just putting it up there, and trying to get some best practices from some actual example.
Then, knowing what is useful and what differentiates, we probably have a dozen, at least, community-level indicator type systems, and probably the states have even more. We don't need to build another one, we need to find out sort of how to get the best out of those, I think. I would like to start that way, from my recommendations, advice and principles about the job we were first asked to do, which is to help HHS, one, get the data up, get it out there and then we will see what happens later. Get it out in a way that can promote and accelerate applications. Some of the ideas today, I think, were fine.
Then, two, maybe this is just the way things have to, how do you accurate the serious applications. I don't know. Again, maybe we don't know, maybe it just has to happen. Maybe there is no theory, maybe it is atheoretical, it is market-based.
Then, the other one, do we want to pick an area. Again, I would ask the committee to come back with an idea. Is it community health tools, is it quality tools, which you probably don't want to go there. Just stay in the health area, but otherwise, is it social determents, is it environmental. Again, this is the information that communities put up, that organizations put up. You are looking for a house in a neighborhood. Some of these apps will tell you the quality of the schools and probably the environment in there. Is there an area that we could look at to kind of specifically apply some of these ideas.
MR. DAVENHALL: Jim, I prepared a one-pager. I don't want to discuss it today, but I want to read it on the airplane. Part of it is me trying to figure out what you have asked me to do, what job you really want us to do. I actually think we ought to spend some time thinking about how we enrich the ecosystem that we are talking about. Good medical advice to be that good, and proper diagnosis before we start to fix what we don't understand. I offer that to try to figure out what it is you really want to do, and at the same time, give you some ideas of what I think would bring the developers to the table.
DR. COHEN: Thank you so much, Bill. That was going to be essentially my ask for our joint homework assignment. I need to understand more what space you live in and what makes sense, and how you use words and language, so that we can provide you with the tools that you need to succeed. What I would like for you to do is understand what information we have in the space we will operate in, and see if that leads to a fruitful relationship.
DR. VAUGHAN: I would also say, though, that part of the answer to what you have asked is when do we get to from the cool to something that is useful, is to start out with a different question. That is, what would be useful to which we would like there to be easier access to the data, that that would empower communities, that that would push this thing forward.
Same thing as any health survey, the same thing as any other epi study, any other public health intervention. To start with, what are your goals and objectives, to sound like somebody who used to beat me over the head with it.
DR. ROSENTHAL: I would respectfully disagree. When I use the word business and usage, I also mean non-profit, as well. I am just using that broadly. I think things have been done for quite a while, saying what would be good water to drink, and actually asking the market, where do the horses want to drink, is at least how I understood how Todd and Bryan framed the problem. I will share my slides around that.
Bruce, to your point, I can share my two cents, I can provide resources that you guys may want to look at. I think the beauty, what we have learned in the consumer web world, whatever you want to call it, is the learning center, as crowdsourcing that question is a really powerful way to go about it.
MS. GREENBERG: I think this has been a really useful and interesting discussion. I think we have gotten some substantive reports from the department, but also some really great little reports from the members. It is Friday afternoon, and I, for one, am exhausted for various reasons.
I think that what Josh presented, and then what Larry said, and I am sure what is on here, Bill, and what Jim said and everything, there is an opportunity to try to connect these things, because we are talking about at least two different things, which would be very helpful to, like Bruce, who is running a state data operation. That is, what are some really top priority things that would make these data more helpful to developers.
Also, that is a different question then, how could we make this data more accessible to people who are not developers, but have all of these questions. We know that health websites are among the most visited websites. Increasingly, when you ask people where did they get their information, whether they have a health problem or whatever, they are all going to the web, we are all going to the web.
That touches on the workforce issue, which I agree with somebody, I don't know who it was, who said it is going to take years to quadruple, and it won't happen probably, but the people who are getting MPHs, and you don't necessarily need a PhD in epidemiology. That is where some of Josh's suggestions, I think, and we need to explore them more, are ways to make the data easier. Whether it be these tutorials or these examples of these learning centers, not just for applications or for people who really are data savvy, but for the rest of us.
NCHS, years ago, we had an applied statistic learning program or something, and it was one of many things in the cooperative health statistic system that was somewhere kept and had new names, others were lost. There was a reason why we have that, and nothing ever filled the gap frankly.
I think that yes, we want to increase the really technical and well-trained workforce. We want to just push this data out to people in a way that just makes it much more accessible. They don't have to be statisticians, they don't have to be graphics gurus. I seem to be resonating with Josh here, so I feel like I am not complete on the wrong track.
MR. SCANLON: The employment rate is almost 8 percent. I think there are a lot of folks who have the skills who can actually help here. I don't think you need necessarily a lot of new folks. I think you have to repurpose.
MS. GREENBERG: We don't need what?
MR. SCANLON: Well, I think there are a lot of folks, including MPHs, who are unemployed. I think there is probably an excess.
MS. GREENBERG: People who are unemployed, who could get up to speed, because they have other skills or they could be repurposed or whatever. Our focus is not just the people who do the apps, yes. It is also several of the things that Josh suggested, I think, are really for leveling the playing field.
I think we will have a transcript from this meeting. We will get your slides, we will have this. Then, those of you who are members of both groups, of the working group, think about how to integrate it, possibly with the project that you have suggested. I think both the full committee and this working group will be more effective and more productive if they can support each other.
MR. DAVENHALL: Marjorie, something you can do to help us with, some of the things that you are mentioning, like, I look at your conference you had a month ago. Most of those sessions with the data geniuses were filled. That tells us there's something that is really working there, but there was no sessions for CMS at that. They didn't have a data guru. Allison wasn't present talking about it or the data guru behind all of that. I am saying that was well-attended. This is the place where you go and meet a person who has the diagram you want.
MS. GREENBERG: We have to realize the environment that we are in, that we have been told conferences like that, which are biennial, should be no more frequent than triennial or quad annual. We have got to look at all of the different opportunities.
DR. CARR: That is why we are here. We each speak different languages. I understood the first part, the second part, not so much. Likewise, I know what an MPI, but I think it is convening these kinds of groups in this small setting, in larger settings. Josh and Ed, having conversation, coming at it from completely different environments. That is what we are here to do. I think that again, our next NCVHS meeting is in November. It would be nice to sort of have a thought about what is the next thing. I mean, I think you are right, we need to reflect on what we heard today. I would like to hear from you guys, what would feel right to you. We have done some presentations of what there is, what would be the next thing.
MS. QUEEN: Who is the audience for this working group? Is the audience for the working group developers?
MR. SCANLON: It is HHS. You heard Bryan say today, and he really did it sort of without substance. It was how do you just promote this generally. I think we are looking for the data that we have, and we can go into a lot of detail, and for the data we have already posted on Healthdata.gov, other mediums, too, how can we promote the use of the data for applications of all kinds. I think he is specifically looking for applications, third-party developers.
Number two, are there other datasets that should be made available there? If not there, should they be available somewhere else? This is clearly an HHS working group. It is not necessarily for developing reports. This is really to give us fairly practical advice on bridging the gap between our data producers here and other government agencies, and the apps.
Again, we could if we wanted to go into the various user communities like public health and research, but I don't think that is where Bryan is actually asking. There are a lot of ways to do that. I think you have been selected because you are technology. You have developed apps yourself, you know the community, you know them. You know what we are not doing.
I think we are looking for not a report. I think we are looking for advice from you, what are the principles, what are the recommendations that you would provide us, I would say just after today, for example. I would like this to be an actual group where, in two weeks, you give us your comments and we take them to Healthdata.gov. I don't want us to revise Healthdata.gov again and get the metadata wrong, so there we are for another three years.
I think I would like us to keep this fairly practical, agile and oriented. What is your sense of advice after today, after you heard from Bryan, after you took a look at the Healthdata.gov, at the Health Indicators Warehouse. Give us some advice that we could take from the department for that. That is number one, because you already are good, you are experts at this already.
Then, beyond that, it would be is there a longer term, are there other issues. Do you want to think of ways to bring the communities together. It may not be the conference, but it may be other ways. It might be Challenge. We're FACA now, so we could probably host the meeting where we literally bring in, not thousands, but folks that we think would represent the points of view and expand our own.
I guess what bothers me is Healthdata.gov. I love it, I support it. It is such a small part of the data we have, that it is what we are making available without restriction. I guess what I would really like you to do is think about how we could take these other datasets that are not restricted access. You can get to them now, but you have to sort of have a data use agreement.
Then, look at those in the sense of, is there technology that can help us, are there apps that can help us. I don't think we are going to be able to just put those on the website without restriction. Maybe it doesn't have to be quite the way we do it now. Maybe there are platforms and technologies.
We have probably six of these research data centers within HHS, and we can describe those to you. Maybe there is a much more facile, agile platform that we could aim for, that would do a lot of the work for the analyst. Then, you get to the point where now you need the data, like the 5 percent sample or the partially synthetic. I think that is really what Bryan and the leadership here is looking for. Not that we don't have broader issues and needs, but I think that is really what this workgroup was set up to do.
DR. KAUSHAL: Can I make a suggestion? Thank you, first of all, that helps clarify a bunch of things for me. I think we need to divide the workday into the framework I sort of started with the supply site. Joshua had some great ideas at best practices. That is not everything, so how do we increase the dataset. I think we should spend some time, thinking around how do we improve the actual supply of datasets and how do we just make it easier to access.
Then, I feel the next piece of our work, and with our collective networks, I think we could help with that, work on the demand site. As an example, I mentor in New York, Blueprint Health. It is Rock Health in San Francisco. All of these young engineers are crying out for data to make their business models and apps better. I also mentor them on business models. I think we could bring them to the table once we sort of got our ducks in a row around the supply side, iterate with them, get their feedback and then maybe go through another cycle.
DR. ROSENTHAL: There is a course at Harvard we are doing on entrepreneurship around that with the young developers this January.
MS. GREENBERG: Could I just suggest that if we have a transcript, we are going to have a draft summary. We also have these other documents, but they will maybe be integrated since they were part of the record. We have the SharePoint site, if we could use that to start teasing out some principles, as you said, some priorities, both for applications and for more general use, for increasing literacy really in the data across the board. Use that as a way to be communicating, and do you want to have a teleconference before the November meeting? It is two months away.
I think there is a lot of content as to what I was saying before, in what we have discussed just this afternoon and before, and that has been presented, that we could start organizing it somewhat in the ways that Jim had suggested and that all of you have said.
MR. SCANLON: I would think by November, we would have a rough draft. We would just accumulate your ideas for principles, and sort of how HHS do this.
MS. GREENBERG: Priorities, if you can do these four things.
MR. SCANLON: This is what would be most useful.
UNIDENTIFIED SPEAKER: Part of it goes back to the goal to increase the number of apps, the goal to expand the universe of users so that the data is more distributed.
DR. VAUGHAN: That is why we do everything, to improve health and healthcare.
MS. GREENBERG: Many people as possible, able to make some use of.
MR. SCANLON: There is an approximate kind of goal and that is get our data out, so that would provide the conditions for these.
DR. VAUGHAN: You will not forbid me to tell any CMS and Ed Sondik and everybody else how they can improve theirs as well, because ultimately, they will do that. You see how it all started with the open government focus and with these ideas of Healthdata.gov. By the way, it is health and environment data. Healthdata.gov includes other agencies.
PARTICIPANT: I think if you want to get beyond a cool app, you have to ask for something beyond a cool app. You don't have to know that it has this button here and that button there. What is the question you are trying to answer.
MR. SCANLON: What is the health or other personal issues? By the way, it is tools, as well. For example, I think NIH just put up an BMI calculation index. We have had others. Those are tools. You decide, you give us a sense of, are they valuable? Are the datasets more valuable? Are they both valuable in just separate things? Then, those kinds of apps get put on all kinds.
DR. VAUGHAN: Does that make a difference in people's health? They get made, but you have to ask, so what? Again, kind of pull back to kind of a more epi framework, what is the question you are trying to answer.
DR. COHEN: The question that I am trying to ask and answer is, how can we use all of the data we have locked up here, to create information to help individuals and the communities improve their health.
MR. SCANLON: Through this particular technology for now.
DR. CARR: What I am hearing is we are going to get all of the information compiled. We will have our transcript, perhaps we could get an executive summary out of that transcript. I have tried to capture and revise, as we speak, what we said here. We will meet again on the second, on November 14th, so between now and November 14th, is there a sense that we would want to communicate, develop, revise, have a call?
DR. KAUSHAL: I think it would be a great idea. We have three hours again on the next meeting? I think we can realistically have time to maybe jump into one of the big issues, however you want to define it. Again, I am open to whatever framework. If it is a supply site, we can focus maybe the next one just on supply, if it is demand, we could do that. If it is supply, I would ask us all to think through.
Joshua has already started with his stack, as well, around what are some of the best practices we have from our combined experiences. Why don't we just lay that all out, maybe on the short side. Hopefully by the next meeting, we would have our compiled thoughts and a couple of pages probably around what we think is the best practices. Then, we can prioritize, okay, what are the most high impacts of these, and use the next meeting to debate that.
Hopefully, that will whittle down to maybe 10, 15 different things. Then, hopefully, we can maybe even implement a couple of those. That is just a suggestion. I am not biased either way. How do we feel about that? We have homework, great.
DR. SONDIK: Let me throw one other thing out that I didn't hear. First of all, I think the focus on Healthdata.gov, and is this a good tool for marketing the data, I think is really important. It seems to me sort of a two-dimensional kind of thing. I don't know, it seems like we need something more dynamic.
MR. SCANLON: I think that we start with that, but I think we immediately get others.
DR. SONDIK: Let me throw one other thing that comes back to the people on the supply side, I suppose, which is do we have a responsibility that goes beyond the metadata in saying, here it is. Is there something else there, because we know the ins and outs. We know what it is not saying. Or maybe we don't, we think we know what it is not saying. Is there a responsibility here? Is there a liability here that goes with putting this stuff up, that goes beyond the mosaic issue which is a concern?
The other thing, though, is that a point that came up many times, you have raised it and Bruce raised it, as well, which is the use side. I think before that first Datapalooza, we had the supply people, one strange group, and then we had this other strange group, which was the developers. They got together at the IOM, at the National Academy of Sciences, and I tell you something really sparked.
There were a few questions, like somebody said something about, well, you know we know the difference by race of the number of people who get screened for something. Somebody said, yes, you really know that? Well, of course we know that. It is clear to us that we know that. The other side had no idea, really no idea, of what was there.
Of course, they weren't thinking about it from a standpoint of what the questions were and the users were. I think another component to this is the user side. When I said before, well, you know, it's not clear exactly to me how people would actually use the survey data, Bruce had semi-infinite number of ideas immediately about that. I think it is really important in this, to crank that in. I feel we are doing a lousy job of marketing the data, and it is not clear to me that Healthdata.gov, a website, is what we need. I think it is terrific to build on, but I think we need something more dynamic.
MR. SCANLON: It will become obvious to all of you that that is necessary, but not sufficient for some of these, and that is why that will be the next.
DR. COHEN: One other comment, I think this endeavor is based on an assumption and a shared belief that data will lead to better decisions, and improve individual and entities lives. That is our underlying assumption. We have got lots of numbers, we have got lots of things out there, and we want to leverage that to improve people's lives. We are looking for strategies to do that. I think we just need to keep in mind that that is our assumption.
MR. SCANLON: I would say that information is necessary, but not sufficient. Other than New York City, you can't force people.
DR. SONDIK: We can prove that it is true.
DR. VAUGHAN: It is not always true in the same way, in the same place, at the same time. You have to measure it. It doesn't mean you have to have perfect measure, but you have to measure.
DR. SONDIK: We know how many people in a county don't know that they have high blood pressure. We know that because we have it really solid on a national level. We can model that, that is an app. Now, how good is that model? That is an issue. That is the kind of thing where we use market forces to try to tell us that. Right now, nobody is marketing that. Nobody is marketing the number of people who have undiagnosed diabetes at this point. Part of that is we don't have the workforce over there to use that information.
DR. COHEN: I am comfortable. I have drunk the Kool-Aid.
DR. CARR: The table has been set for a robust and exciting discussion, based on the entries onto the SharePoint. Thank you all very much. I look forward to our next meeting.
(Whereupon, at 5:04 p.m., the meeting was adjourned.)