State of the Science Employment Policy & Measurement RRTC February 12, 2019 National Academy of Sciences Auditorium 2101 Constitution Avenue, NW Washington, DC Communication Access Realtime Translation (CART) captioning is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings. This transcript is being provided in rough-draft format. www.captionfamily.com [CAPTIONER ON STANDBY] DEBRA: So for everyone here, in person, I just wanted to let you know, we're having a slight technical difficulty with our online streaming; so we're going to hold off starting until that's resolved. (Pause). (9:07:56 a.m.) (9:31:11 a.m.) >> These microphones pick up a lot. We can't even whisper up here! [LAUGHTER] >> It's true, we're, like, trying to tell secrets... DEBRA: Welcome everyone to our state of science conference my name is Deb Brucker and I'm from the institute of disability at the University of New Hampshire and the principal investigator of the disability research and Truman center on employment policy and measurement. I would like to thank the University of New Hampshire team for coordinating this event. I would also to extend our thanks to the national institute of disability independent living and rehabilitation research NIDILRR, and our project officer for their support of our center. Our program today highlights research from our center as well as other employment-related research, funded by NIDILRR. I would like to specifically thank our center's research partners for their continued contributions, and their assistance with their Conference. The association of university centers on disabilities, Kessler Foundation, Mathematica policy research and the university of Chicago. We're thankful for the participation of other NIDILRR-funded grantees in our event today as well. We have people attending in person in D.C. and online today. Registrants include people from advocacy organizations, the media, federal, state, and county agencies, research organizations, academic organizations; rehab professionals; and others. Thank you, all, for being here. If you've ever wondered about the effects of Medicaid expansions on the employment of individuals with disabilities, we have a panel for you today. If you're interested in opportunities to expand employment among people with disabilities, stay tuned! If you're interested in trends in employment and the latest employment rates among persons with disabilities, we'll share that information as well. We'll also share information about employer practices, job quality, and contingent employment. Lastly, if none of that piques your interest, we can have fun seeing which presenters can actually stick to their allotted speaking times. [LAUGHTER] >> And make sure that they adhere the guidelines, it's my pleasure to introduce Dr. Phil Beatty from NIDILRR. He serves as the associate director of the office of research sciences, within NIDILRR. And in this capacity, Dr. Beatty oversees NIDILRR's broad portfolio of disability and rehabilitation research grants. Prior to joining NIDILRR in 2003, Dr. Beatty worked at the national rehabilitation hospital center for health and disability research. Conducting research on the health and long-term care experiences of people with disabilities. Prior to beginning his public service at NIDILRR, more than 15 years ago, he authored or coauthored more than 20 peer-reviewed publications on the experiences and outcomes of people with disabilities among other topics. He has a Ph. D. In sociology from American university. [APPLAUSE] PHILLIP: (Approaching) thank you, deb, and welcome to everyone, and -- and thank you, especially everyone at the research and training center for pulling together this important event. The NIDILRR team, and all of our colleagues, across ACL, and ODEP, and other critical government agencies, and the disability community -- as Deb noted, we're all looking forward to learning about the current state of the science. Related to the employment status of people with disabilities. In the extent to which employment outcomes are related to specific policies, practices, interventions that are -- that are happening in the field. The whole agenda today, as Deb, kind of foreshadowed. The whole agenda is packed with presentations and discussions that are central. To understanding employment of people with disabilities. And, again, the policies and practices that can drive employment outcomes. Now -- and I'm going to do something similar to what Deb did: Just reflecting on the agenda here and the real value in the agenda and the way it's put together. On panel 2: Later this morning, I'm interested in learning more about the current gig economy. And -- and how that might be related to job quality among people with disabilities. Over lunch: And in other sessions, we're going to -- there's going to be a lot of discussion about measurement of employment rates among people with disabilities. And trends in those rates over time. The monthly nTIDE reports have been a great addition to knowledge, in the field, just really timely addition to our knowledge. And we're looking forward to hearing more about the present, and the future, of those reports, and what they mean to us. This afternoon, across two panels, we'll hear about a variety of interventions and opportunities, and service systems and incentive programs aimed at increasing employment rates of people with disabilities. And, again, all of us here gathered here in person and online, are -- are looking forward to learning from those panels, and participating in discussions with the panelists. So, as Deb noted, there is a panel on -- the first panel, and the topic, one of the central topics, on this, on this panel is "the extent to which Medicaid expansion decisions happening in the states, are -- related to employment outcomes for people with disabilities." And this is the panel that I've been looking most forward to. It -- because it's a big question. And because we have different grantees from NIDILRR's portfolio who have been looking into that question. About the relationship between Medicaid expansion and employment rates. And there is a rich body of knowledge that NIDILRR and NIDILRR's grantees are generating around this question. For those of you who don't know: What was scheduled to be presented, I'm going it give away the punch line upfront and let you know that the two groups, of NIDILRR grantees have come to different conclusions about the question of Medicaid expansion and its relationship to employment outcomes. And in the world of policy research, this kind of thing happens all the time. What I'm really looking forward to is the collegial sharing of information. And ideas about samples, methodologies, findings, interpretations, and -- ideas about future iterations of these -- of these analyses. From our perspective in NIDILRR, the path to better knowledge, and better knowledge, based policy is through triangulation, collegial methodological discussion, and multiple points of data. And to us, and, I think, to all of you, that's a -- that's a refreshing, and affirming approach. Especially, in the current overheated, political environment. That we're right in the middle of here and on constitution Avenue in Washington. Today. So with all that being said, I've learned this morning that one of our panelists, Dr. Kennedy, is -- is snowed in, in the upper northwest, so he won't be able to be here to represent the findings and the analyses that -- that the CHRIL has been doing on this topic. And -- and that's too bad. But it just means that we won't be able to see firsthand that discussion; that is ongoing; and that will be happening in the coming weeks and months. But we trust that that discussion is happening and through triangulation and collegial work, we will be able to... triangulate and figure out the best path forward around this important policy question. So, again, thanks to all the panelists, on the agenda today -- the research that you've been doing; and the discussions that we'll have -- are really the foundation for more effective Policy. To drive up employment rates for people with disabilities. So I'm going to turn it over to Andrew, who's going to moderate our first panel and, again, welcome to everyone here in the room and those of you online! Thanks, Andrew! [APPLAUSE] >> Andrew Houtenville: Thank you, I'll be very brief. Jody is going to talk about the trends in health insurance. Purvi is going to talk about trying to identify the impact of a program, and one of the most creative things as a social scientist that I do, whenever I can do it, is to find the impact of Policy, without the luxury of experimental design; and so what Purvi is presenting and what Jae would have presented is a way of thinking about trying to identify policy impacts. And it's generally -- natural experiment designs, you know, taking advantage of natural variation. And difference -- and difference analysis is typically the way people will handle that. And it's one of the most challenging things to do and there's -- when you have differing results, there's a need for replication, and getting more data sets and more data sources. The declining employment rates of people with disabilities, in the '90s and 2000s, was a major, major issue; and we lots of work replicating that with as many data sources as possible and eventually came to some reasonable conclusions. And that's kind of where we are as a science today: Replication. Using as many data sources as possible. Carefully looking at methods and sampling. But you'll see one side of that, Jae's slides will be available online, I believe; and the paper is -- some other work is published. So I'll turn it quickly over to Jody. [APPLAUSE] you don't have to clap for the moderators. [LAUGHTER] >> Jody: Hello. I'm trying to figure out if I can advance slides. There we go! There are some restrooms, wi-fi. [LAUGHTER] >> Jody: (Continuing) does anyone from UNH know where the slides are. Oh! Somebody is moving it for me! Look at that. Great. Thanks. With the introductions been given I feel anticlimactic and I'm going to be presenting trends in health insurance for workers and I believe Purvi can talk about the employment findings, this is joint work with Anna hill looking at the trends and work we're continuing right now also look at watt the trends in health insurance mean for access to care for workers in disabilities since there has historically been gap in difficulties accessing medical care for that group. So before I turn to our results, I think it's worth stepping back and thinking about the landscape for health insurance that existed in this country before the ACA went into effect in 2010, public health insurance was available for many with disabilities if they were receiving SSDI Social Security Disability insurance or SSI, supplemental security income. Medicare was available to those receiving SSDI after a 24-month waiting period and in a lot of states Medicaid was available to those receiving SSDI. Some states had separate application but in general they also received Medicaid. There is other coverage available through the VA and TRICARE although that was relatively less common than insurance that came through Medicaid or Medicare, but the predominant way for accessing health insurance for people with disabilities and people generally was through employer sponsored arrangements if you had a job that offered coverage you could obtain health insurance that way. But that was available for people only working in covered occupations. If they were in an industry that tended to not offer health insurance, working part-time working sporadically health insurance wasn't available often through an employer arrangement and what that left for people who didn't have public health insurance or an employer provided option was nongroup coverage that coverage tended to be prohibitively costly in many cases for people who had preexisting conditions it wasn't available at all with workers with disabilities and adults with disabilities more generally there were limited options available for coverage before the ACA. Starting in 2010 the ACA really changed the landscape so immediately after the provisions went into effect in 2010, there was a removal of preexisting conditions, limits that allowed more access to coverage. There was a removal of lifetime benefit caps and for people with disabilities, these were pretty salient; that they had more coverage options available to them. Extended dependent care was available until age 26. Prior to the ACA, children who were covered on their parents' employer plan when they reached age 18 would no longer have access to the coverage but the ACA extended that until age 26 which was really important for children aging into the work years to be able to maintain that coverage. But the real policy changes that extended access and affordability came in 2014. Medicaid expansions were available to individuals with incomes up to 138% of the federal poverty line. Although those expansions only occurred in about half the states in 2014. Since then another dozen or so states have expanded Medicaid. So in many states Medicaid was available to the working poor, regardless of disability status, which is a real change from what was available prior to that time. And in addition in the same year, health insurance marketplace allowed people to access group coverage by essentially shopping online for those plans in some states those were state based marketplaces in other states they were federally facilitated marketplaces but it allowed people to get coverage in ways they hadn't been able to do before. In addition to that, there were premiums subsidies available to people earning up to 400% of the federal poverty line, this really opened up a range of health insurance options for both people with disabilities and people without disabilities. In 2014 there was also an individual mandate, for people to have coverage although our working hypothesis is that for people with disabilities, this mandate really wasn't what was driving their decision-to-seek coverage; that this was a group that has chronic health needs and health insurance was something they wanted regardless of the mandate. So I should note that Jae Kennedy who, obviously, can't be here today has done work in this area with his coauthors Lex and Liz, to look at trends in health insurance for adults with disabilities in general, we focused our analysis specifically on workers with disabilities for a couple of reasons: One this is a group that may earn too much to be eligible for federal disability benefits. So SSDI and SSI have income limits related to substantial gainful activity and beneficiaries historically have noted they're really fearful of losing benefits, for people working above that level they likely don't have access to Medicaid or Medicare through that channel additionally many workers with disabilities work in part-time or nonsalaried jobs or in positions or industries that aren't included in health insurance coverage from an employer and this is becoming potentially more important as contingent or gig work -- jobs become more common and so this is a group that may lack access to public benefits and may be limited in the availability of employer sponsored coverage. And so the availability of this coverage through the marketplace our through Medicaid may have been really salient for this group and I think it's important to note really what happened with the ACA was there was this -- previously there was a real strong linkage to receive healthcare the most likely -- health insurance the most likely way to do that was through employment and the ACA changed that. So we answered three questions. The first is how did the share of working age adults who had health insurance change from 2001 through 2017 so we look at workers with disabilities from over this period and we look at how many had health insurance at all. And second we looked at how the source of coverage changed among this group so we really sought to understand to the extent that we see changes in insurance coverage overall what's driving those changes and to try to benchmark what our findings mean relative to other groups we considered the changes relative to adults with disabilities who are not working so this is a group that's similar in disability status, but different in their work status. And then workers without disabilities. So this is group of people who are working, and they have access to employer sponsored coverage, in the same way working adults with disabilities may. But don't report having a disability. We use the national health interview survey I call it the insurance survey because clearly we had insurance on our mind. And we used the IPUMS health surveys version which offer a nice harmonized data series from 2001 to 2017 we can follow through measures of employment disability and health insurance. We used a measure of employment as whether or not the person identified working for pay in the past 1 to two weeks. For disability status we used an affirmative response to a question of do you have health condition that limits the quantity or type of work you have? I know there are many measures of disability this is the one we're using for the study. In part because the national health interview survey did not have a -- we were going to use the six-question survey from the census but they didn't start asking the series until 2010 it didn't really allow us to look before the ACA. There's, obviously, this measure is imperfect but given we're looking at employment this is a reasonable measure of people who think they have a barrier to work. In each year of the survey from 2001 to 2017 we have about 1400 workers with disabilities. Each year, among the full sample, about 10 to 15% have a disability and we see employment rates for people with disabilities that are 24 to 25% and for people without disabilities it's about 78%. So it's relatively consistent with information from other national statistics. We look at health insurance that's at the time of the survey so it aligns pretty closely to the measure of work which is over the past 1 to two weeks and there's, obviously, a myriad combination of health insurance but we try to look at big combinations both for ease of presentation but also because these are the ones that are probably most salient for the policy changes that occurred around 2010. So we looked whether or not any person had any Medicaid this includes Medicaid alone or in combination with any other source of coverage. We looked at those who had Medicare but didn't have Medicaid so this is Medicare -- they can have other options a as well but just not with Medicaid. If they had both, they would be in the Medicaid group. We looked at coverage employer sponsored purchased only through an employer without other coverage and finally we have a catchall category we call other or largely privately purchased. This includes people who have previously purchased plans which could include nongroup coverage group coverage, we also have in this category a small number of people who have other public coverage this could be TRICARE or VA or combinations of coverage that aren't captured above. The bulk of this group, however, really does seem to be privately purchased we think may reflect marketplace type coverage. 5 months all right and uninsured if none of the coverage sources. So first I want to show that this is the rate of uninsurance and we see there's a real decline in the share of people who are lacking health insurance coverage and this change really occurs after 2014. So the two lines here denote both when the ACA went into effect in 2010, between 2009 and 2010, and then the bigger policy changes with Medicaid expansions and marketplaces in 2014. I think notable is prior to 2014 we saw uninsurance rates bounced around a bit but hovered between 17 and 22% and then following the implementation of the policy changes in 2014 we saw immediate and marked decline in the share uninsured from about 21% to 11%. There's a slight up stick in 2017, which could reflect the small sample size. But it also may reflect changes were going on with the enrollment period and other things that may have really meant this was a change in the share and I think it's worth tracking that going forward. This is actually the opposite of what I just showed you the share insured so we said sort of there was a decrease in insurance and noninsurance this is for workers with disabilities and I show thus because I also want to look at the group, who the nonworkers with a disability and the rates of coverage among -- are actually higher reflecting they had more access to Medicaid and Medicare, here's the line for... workers with -- without a disability. And we see their patterns roughly the same and overall all these groups have similar increases in insurance coverage following 2014. And finally, here is the group of people who are not working and didn't report having a disability. They have rates of coverage that are lower because they lack access to employment and public coverage but we see for that group the increases in coverage that came after 2014 were actually relatively large. So among workers with disabilities, who had insurance coverage we sought to understand what types of coverage they have. Here we see that 8.7% had Medicaid coverage before the ACA went into effect. There's a slight increase by 2014 perhaps reflecting the fact the great recession caused more people to seek federal disability benefits and that could have driven some of this but by 2017 we see a doubling relative to 2013 in the share who have Medicaid. We see a slight uptick as well, from 13 to 16% and I mentioned earlier there's other things in this category the bulk of the increase that we see here seems to be driven by privately-purchased coverage. We also see a decline in the share of workers with disabilities who have Medicaid or sorry who have employer sponsored coverage. So it was about 52% had employer sponsored coverage in 2009, it fell slightly by 2014 which might have reflected sort of a trend over time, in the share of employers who are offering coverage but by 2017, after the other ACA changes went into effect it was down to 44% and I'll come back to talk about why that may be. We didn't see much of a change over time in the share who had Medicare, and, again, for completeness this is the group that's uninsured that I have showed previously. So we wanted to understand really how the compositional changes I just mentioned the increase in Medicaid and... privately-purchased coverage related to what we see for other groups. So since I have two minutes, I usually talk quickly and never run out of time. So this is workers with disabilities, we see during this period we looked over three periods 2001 to 2009, 2010, to 2013 and 2014 onward. We see that for workers with disabilities, there was a real change in among those who had insurance the share who got their coverage from employer. The share who got employer fell from 61 to 2.warn, at the same time there was a notable increase in Medicaid and previously purchased coverage I mentioned earlier here are workers without disabilities, we see also there is a shift away from employer sponsored coverage, but the shift is much smaller. There's also much higher rate of employer sponsored coverage before the ACA went into effect for this group. And finally, nonworkers with disabilities, this group had a much higher rate of Medicare and Medicaid coverage, many of these were receiving federal disabilities benefits and we see there's a shift away from employer sponsored coverage. That said, in both absolute and relative terms, the shift away is largest for workers with disabilities. So just to summarize: And I'll talk about the implications in a second. There was a decline in uninsurance for all groups that's a good news story across the board. Notable increases in coverage for Medicaid and previously purchased sources for adults with disabilities and other groups to a lesser extent and there is a decline in employer sponsored coverage that seemed to be larger for workers with disabilities, than it was for other groups. And I guess really important to note here we can't term this crowd out. So there's always interest in understanding the extent to which people substituted out of employer provided coverage once public coverage is available, we have cross sectional data. We can't observe what individuals did over time and more importantly we don't know which coverage they were eligible for. There's evidence in the literature suggesting that employers weren't reducing their offers of coverage during this period but we really can't say at the individual level this was substitution. We can just say there was a change over time. But I think it's important for workers with disabilities to understand what this might mean. So first, there was an increased availability in coverage and the first thing is this coverage wasn't connected to an employer and wasn't connected to disability benefits. So the extent that people were able to get coverage without accessing federal disability benefits meant they potentially had reduced disincentives to work so if SSDI and SSI meant people were less likely to work the availability of coverage from elsewhere was really important. At the same time there may have also allowed workers who were working, to potentially change the work behavior so there could have been a reduction in full-time job block; so to the extent prior to the ACA, workers with disabling conditions, kept working full-time so that they could keep their employer sponsored benefits, if this allowed them to reduce from working full-time, to manage their medical conditions or work at a level they wanted to work, this actually could be important as well and, actually, in our data we saw, some change in the share who were working part-time. So among workers we did see an uptick using the NHIS and had the share working part-time. I don't think that's definitive but worthy of additional study and I think most importantly in addition to having increased access to coverage options there's the possibility that Medicaid coverage actually offers a better set of covered services for people with disabilities. Jack, and Alexis did a study about this in 2015 and they found workers with disabilities identified that being able to access personal assistance services and durable medical equipment through Medicaid actually may allow them to have better health -- better health improvement and work more than they would have been if they were -- using the services provided from an employer plan that often don't cover that. So I think overall it's a good news story, definitely interesting to look at the shifts in coverage. With that I will thank the NIDILRR and the EPM RRTC for funding and I will stop! Thanks. (9:58:52 a.m.). [APPLAUSE] >> Purvi: Uh-oh! All right. Great Hi, everyone Purvi Sevak from Mathematica policy research. I'm going to be talking about tattoo the Affordable Care Act Medicaid expansions and the effects of people with disabilities, this is work I have done with Jody Schimmel-Hyde, who you just heard from. Okay. So as Jody just described in detail, before the Affordable Care Act on Medicaid coverage for adults with disabilities, was lamented to certain groups. Specifically, those receiving disability benefits through the supplemental security income program SSI, it was also available to those that were deemed medically needy. Which means that -- the medical expenses that exceeded their income. Starting in 2014 with the ACA, states were now allowed to offer Medicaid coverage to those with incomes up to 138% of the federal poverty level. At the end of 2014, 25 states in the District of Columbia had expanded Medicaid. And as of now, 37 states have adopted the Medicaid expansions, the last two were Utah and Idaho which passed it in 2018 election, it's important to note that just because a state has expanded Medicaid, doesn't mean that coverage -- is actually been offered. One notable example is the state of Maine, which in 2017 voters voted to expand Medicaid. But the governor refused to implement it. So new governor was elected in 2018 and she authorized implementation as of January 1st of this year. It's also important to note that all states had expanded Medicaid don't offer -- there's a lot of differences across the states. So it's not the same everywhere. (Pause). >> Purvi: How might Medicaid expansion affect the employment of adults with disabilities? Jody covered this pretty well. But I will just mention, that, you know, Medicaid is a health insurance program. It wasn't intended to be an employment policy or employment support. Nevertheless, for all the reasons Jody mentioned, there are reasons to think that expansion of Medicaid availability of Medicaid coverage could afford -- could affect employment rates of people with disabilities. Specifically, Medicaid expansions are now offering coverage to the working poor. So, this can have an impact on employment in either direction. It could increase employment rates of people with disabilities, if it reduced barriers to work. In addition, because it delinks Medicaid coverage with SSI, it could basically reduces the attractiveness of SSI just as a source of Medicaid coverage. So it could increase employment because of this. But expansions of Medicaid could also decrease employment. If it means that this public coverage allows people to re- -- to -- to leave their jobs that they were staying on, for the insurance coverage. So-called job lock. (Next slide) so the literature in general, there's been dozens of papers on the effect of Medicaid expansions and in general they find that, coverage expanded. There were improvements in access. And you saw some of this in Jody's presentation in terms of increases in coverage, but studies that have looked at employment effects have largely found no significant effects. The studies have looked at employment rates, labor force participation, job transitions into retirement and hours worked. So these are studies that have looked at the full population, not the population of individuals with disabilities. There's one recent study by hall and coauthors that Phil alluded to that shows a slight increase in employment rates for people with disabilities and expansion states relative to nonexpansion states. They are estimated increase in employment is limited to 2017 and earlier study that they published found no impact in the earlier years after the Medicaid expansions. So how can we assess the impact of expanding Medicaid? As Andrew mentioned, we can think of the Medicaid expansions, as a natural experiment. Perhaps. Oftentimes when policies are implemented, they're implemented differentially across states or across local areas. And while the ACA was meant to have Medicaid expanded, throughout the country, we know that some states elected to not expand Medicaid and from a research point of view that offers an opportunity to study what the impact of the policy change was, because you can compare outcomes in states that expanded coverage to those that did not. This is really possible only under some strong assumptions. That the states that did expand Medicaid weren't different in substantive ways from the states that did expand Medicaid. And we questioned whether this is a reasonable assumption. So one look at the map of states that expanded Medicaid, and states that did not expand Medicaid really reveals, this is sort of an in general red state blue state stories, so these are very different. This map shade states based on their expansion status as of 2016, the blue states are states that expanded Medicaid in 2014. The red states, are states that did not expand Medicaid. The white states -- I should say that did not expand Medicaid by 2016, and we look at 2016 just because of the data we had available when we started the study and the white states are the states that expanded Medicaid later and also Massachusetts is shaded white and these are states we don't include in our analysis. A closer look at the data, shows measurable ways in which expansion states differ from nonexpansion states. Specifically, before the ACA, relative to nonexpansion states, the states that expanded Medicaid, under the ACA, had lower poverty rates. Higher average incomes. Higher population density. A higher share that already had Medicaid coverage. And a lower share of people that were uninsured. These states had slightly higher rates of employment for people with disabilities. However, they also had slightly higher unemployment rates overall for the full adult population. These differences are important to recognize because they're likely to be correlated with employment rates in the period after the Medicaid expansions as well. So the innovation of our study is to address the fact that the group of states that did not expand Medicaid can't just be used as a comparison group in full. We know we need to have a comparison group if we want to look at the impact of the Medicaid expansions. We can't just look at the states that expanded Medicaid and observed what happens, relative to the preACA period. Because we know that there is a lot of things in the macro economy that were changing. We had expansion from a big recession. So what we do is we take the comparison group of states that did not expand Medicaid. And we break them up into smaller geographic components. And the logic behind this, is that let's say we take a state that expanded Medicaid like Maryland. We might not be able to find a perfect twin for Maryland that did not expand Medicaid; but if we break up Maryland into smaller units, counties perhaps, we might be able to find reasonable twins, for those counties among all of the counties in the states that did not expand Medicaid. And so that's what we do: We don't use counties; we use construct called census public use microdata areas. Because that's a geographic unit, a substate geographic unit that's available and publicly available census data. We use the 2010 to 2017 American Community Survey. And it's not in my slides but I should also note how we measured disability. The ACS has a six-question series, to identify functional impairments in the population. It does not identify everyone with disabilities. It wasn't designed to capture everyone with disabilities. Andrew and others have a number of published studies that show how well it aligns with other measures of disability but it is, nevertheless, the largest sample of data that's nationally representative, that covers all the states, and the -- the -- the disability measure that's in there, is the measure that we're used to seeing, in the monthly employment reports that we'll hear about later today from nTIDE. Okay. So how well do we do with trying to deconstruct the states so that we can have a good comparison sample? I'm going to illustrate a couple of examples. So this slide has two graphs: The first graph illustrates on the left, the trends in unemployment rates in the preACA period. So 2010 to 2013, the blue line is showing the unemployment rate and expansion states in the years before the Medicaid expansions. The red line is showing the unemployment rates in nonexpansion states in those same years; and we can see on the left that the unemployment rate is consistently higher, in expansion states relative to nonexpansion states. And the preACA period. On the right, we plot unemployment rates but now, we do this using.... um, a related sample of the pumas in the expansion and the nonexpansion states and we reweight them so that the geographic areas are similar in terms of their characteristics. Like the unemployment rate. And we can see that after we do this statistical relating, the trends in the preACA period are pretty much identical between the Pumas in the expansion states and the Pumas in the nonexpansion states. I'll show you one more example. Here we're looking at the percentage of the working-age population that has income under 138% of the poverty level. We look at 138% because that's the -- the eligibility threshold for Medicaid coverage in the expansion -- under the expansion. So, again, on the left... we can see that the poverty rates are very different in expansion states than nonexpansion states in the preACA period where nonexpansion states have notably higher poverties rates compared to the expansion states on the left we plot the poverty rates in the statistically related pumas, and we can see that they're much more similar. So taking these reweighted sets of Pumas we look at individuals, in the expansion and nonexpansion states and we calculate the change in the employment rate around the ACA, in nonexpansion pumas. This serves as a counterfactual of what would have been expected in the absence of the policy change. We make the same calculation in expansion pumas and use multivaried analysis to control for individual characteristics and puma attributes like industrial composition that might be different across the states. So what did we find? Well, we don't find any good news, but we also don't find any bad news, what we find that's positive is that employment rates among individuals with disabilities, rose in the post ACA period across all states. This has been documented in a number of places, including the nTIDE reports that we see every month. , however, we don't see any significant difference in this increase in employment rates between expansion and nonexpansion states, again, in this sort of statistically reweighted expansion and nonexpansion states. We thought there might be some differences across subgroups. You know, maybe it matters for men but not for women or it depends on how old an individual is, or how many disabilities they have. So we estimated, the same model on a variety of subgroups by gender. Number of disabilities. Educational attainment. Age and a number of preACA state policy characteristics. And we didn't find significant effects on employment for any of these subgroups. So why don't we find any impact? It's possible that the Medicaid expansions did not have an impact on employment rates. The Medicaid expansions increased Medicaid coverage, they improved access, they had a number of very important positive impacts. But they -- the -- you know, the program wasn't designed to affect employment. It's possible it didn't have an impact on employment. It's also possible the expansions did have an effect, but we weren't pick it up in our data and study. One possibility is the expansions had a positive effect in some states and a negative effect in other states and on that we're seeing zero impact. It's possible that the employment impacts of Medicaid coverage may take longer to materialize. This might be particularly true because of uncertainty about the ACA's possible repeal. Nevertheless, overall any concern that people with disabilities would stop working because they now have this public option of Medicaid coverage, seems to have not been borne out, at least not in the data that we have and the study that we've done. I also want to acknowledge funding from NIDILRR under the EPM-RRTC. And... I will stop there. (10:12:54 a.m.). [APPLAUSE] >> Okay. I'm getting a sign we have eight minutes! For questions I believe we have the ability-for-people online to ask questions is that true, Anna? Wave your hand? Anna is saying, yes, so if people have questions, mics are here, we'll have people hand out the mics so please raise your hand. If you have a smart question, Dave Wittenburg, put down your hand! [LAUGHTER] >> Sally? Q. Curious, being from Massachusetts, why -- I think Jody in your slide you said that you excluded Massachusetts, not because it expanded Medicaid, but perhaps because it initiated something like the Affordable Care Act before? A. Sure, I can address that. So the identification -- the way we identify an impact in our analysis, is by looking at pre and post. And comparing states that had a policy change to those that didn't. And because Massachusetts had a lot of changes before the rest of the country, we really -- can't identify -- like, using them in our sample, would... would not help us estimate the impact of the -- the Medicaid expansions under the Affordable Care Act. Q. I think others have done that using earlier data on Massachusetts in particular, but we didn't -- we just took states that expanded in 2014 and we could have done it a different way, where we took all states with the date they expanded and looked pre/ post and included the later expanded but we just didn't include it for ease in looking at the same group of states in the same year (Jody)? Q. H. Dave Stapleton. So I think there's been some earlier research that found positive effects of the ACA expansions on employment among people with disabilities. Is that right? And if so, how do you reconcile your findings with theirs? A. That's the Jae Kennedy paper that was going to be presented. So, they found the positive effect from 2013 to' 17. Do you have any thoughts on -- on how your studies may differ or whatnot? >> Purvi: Sure so... as I mentioned in my presentation, there is a study recently published by Jean hall and coauthors, that documents a positive impact on employment rates of people with disabilities in the states that expanded Medicaid. Relative to the states that did not expand Medicaid. So we've been talking with -- with the authors of that study trying to figure out why our results might differ, and while we haven't quite figured it all out. We have a couple of hypotheses, the first is the fact we use different data. So we're using the American Community Survey. It's a very large sample. The measure of disability. And the employment rates are ones that have been studied, benchmarked, um -- it's possible that we're capturing a different population of people with disabilities. Than they are capturing in their study. Neither one is correct or incorrect. The ACS measure of disability has been criticized by many. [LAUGHTER] >> Purvi: But it is -- you know, it's the data that we use and they have a different data set and a different definition of disability. So that could drive a lot of the differences. A second reason that we might see differences is just differences in approaches, so as I described in our study, we don't use all of the people that live in nonexpansion states. As a comparison group for counterfactual. And I think we explained pretty clearly why we would be concerned with using that full population as a comparison group. So we chose an alternative approach, where we try and align, the expansion and nonexpansion states so that they're similar, in demographic and other characteristics. It's possible that that could account for the difference. I don't think that that really is what's counting for the difference, because Jody and I reestimated our results where we don't do this; and we, similarly, found no effect. So -- the point estimate was a little bit larger, but we didn't find significant effects, even if we go with the more traditional difference, indifferences without accounting for the fact that the comparison group is very different. >> Any questions online yet? Okay, don't be afraid to ask questions, people online! Dave Wittenburg has his hand up, upfront. There's -- Q. Chris from the Department of Labor. I was wondering it's actually -- you answered it a bit in your previous response about the matching. But I was wondering, what characteristics did you match on, and also, what kind of impact did that have on the overall sample size? >> Sure I can address that. So we didn't actually use matching, which we often use something called propensity score matching in these nonexperimental studies. We used an approach called inverse probability weighting, which essentially gets you to the same place but you don't actually have to match people you just create weights that bring you to the same result. (Purvi) it didn't have an impact on our sample size because we didn't have to drop any cases for not being matched, you know, individuals just got weighted higher or lower, the characteristics we matched on, we started with a big long list, and then we had to narrow it down to the things we thought were most important. We used the employment rate of individuals with disabilities. In the preACA period: 2010 to 2013. We used the annual unemployment rates in the area. For 2010, 2011, 2012, 2013. We used population density. And.... another thing -- we may have used industrial composition. I can't remember right now if we did end up keeping that or dropping it. So it wasn't a laundry list of things. Everything else got controlled for in the regressions. >> The idea was to find people -- (Jody) the idea was to find people that resided in similar geographic areas, rather than match them on their individual characteristics. It was -- if you're a person that lives in puma expansion states we wanted to find people and weight people in places that were similar to that person. And so it was really trying to get PUMA-level characteristics of similar geographic areas so that they were facing what we thought were similar labor market conditions; and similar to what you've probably seen with propensity score matching we used standardized measures to look at how well they were balanced, both before and after the policy change. Your last question was how much a difference it made in the estimates? And I think it was about a 25% reduction in the.... point estimate? I think it went from about -- I mean, it was insignificant, so it's sort of hard to say that but I think magnitude wise it was about a 25% reduction in the point estimate when we did that matching. Q. Was it worth it? A. Yes. Q. Okay. [LAUGHTER] A. I feel very strongly our comparison group is worth it. But, yeah -- I mean, you're right. Is it worth it? I think methodologically and conceptually. >> So I have graduates that are learning right now from this kind of methods. And it's always wondering, is it worth it to cut the extra data and take the extra time? >> Even if it doesn't affect the results in this case, it affects the believability of your results. So when someone says, "Oh, we found this, but I don't believe you, because, you know, Texas is not a good comparison for California". >> Right. There's one -- there's a criticism of these kinds of things with policy indigeneity you're picking up they're a very different state systematically because they chose you -- I know the answer is no, conservative versus liberal in the trump voting -- because you had red state versus blue state but it really wasn't. >> Jody: Frankly mostly that was for the colors on the map. >> Yeah, I know. >> Jody: It was not intentional. We did that for compliance, 508. But it worked out that way. >> Red and blue is 508 compliant. >> It passed our compliant check. >> All right. >> Thank you so much. We have ten minutes now for a break, and so feel free to leave the auditorium and we'll reconvene at 10:30. >> For people on the phone, we'll play music and we'll do some impressions. [LAUGHTER] (10:21:40 a.m.). (10:32:55 a.m.). >> Are we ready to start. >> Andrew: Oh, yeah. >> Okay, hello, everybody and welcome to session 2 here. I'm just going to give people -- I'll give people a few seconds, Andrew. I'm sure as soon as they hear you start talking, people are going to be flocking back. >> Absolutely! People at home can't see the crowds coming in, because they're hearing my voice. >> Welcome to the second panel. We're going to follow up the health insurance panel with a discussion of employment trends, quality, and type. And this is -- a -- a really interesting panel, because it takes us beyond the question of, you know, just how many people, are employed. Sort of that zero-1 binary indicator of are you employed? And it gets really at different elements of job quality. And the -- the panel session today, is going to -- has three panelists. Andrew Houtenville is going to talk about long-term employment trends. So are people staying in the labor force more than a single period? Nathan Moon, and Frances Harris will talk about the contingent employment of people with disabilities. And then, Megan Henly will talk about job quality: Tying it all together. And this work really matters, because one of my favorite quotes from a colleague, was Rich, and he once said "work is good" it's a simple statement but very much a true statement, and as everybody thinks about why they're here today, people are here at work and that work develops multiple different aspects of your life. Whether it be from Health, whether it be from social connections. Whether it be from all other aspects of your well-being. So looking at these different aspects of employment, is going to be really, really important. And, so, with that summary, let me turn it over to you, Andrew, to talk about what are the long-term employment trends of people with disabilities? >> Andrew: Okay. Great. (10:35:08 a.m.) All right. Thank you. This is work that I've been doing over the last few years in increasingly over the summer, with graduate students at the University of New Hampshire economics Ph. D. Program Bryce Stanley who can't be here today because he's actually teaching my course right this very minute. [LAUGHTER] >> Andrew: And, actually, he's presenting -- teaching students how to do some of the data extraction work that underlie this -- this work. We're shooting to have a paper ready by the end of the semester. This work may end up being a chapter in Bryce's dissertation, we're not sure quite yet. So let me kind of -- similar to what Jody did, when she talked about the trends in insurance and then Purvi and Jae if he was here would have gone on to talk about an impact of a program on insurance and employment and things like that. I'm going to give the kind of overall lay of the land and the others will talk about quality and different aspects of employment. Um... So this is the -- this is what we've been tracking. Some of you may be familiar with our nTIDE -- the gray -- on the screen (referring to Figure 1). The gray is the great recession, which is defined by economic growth. Changes in GDP. And then full employment, which is defined, both are defined by the national leader of economic research and the full employment is typically around below 4%. It varies a little bit, depending on the author. And so this is the nTIDE result. And what you see is we're actually seeing -- this is the employment of people with disabilities. What you're seeing is actually an unprecedented growth in the employment of people with disabilities. Rich did and I for many years, did research on the employment patterns of people -- employment trends of people with disabilities, and this -- this last few years -- and this is midyear -- continued to go on for a little bit after this -- at the end of 2018. This was really unprecedented and so much so Bryce and I are basically trying to investigate why is this different? Right? If we can figure out why is this different? Is there a sector, a type of job? And we'll eventually get into not necessarily job quality. Although we will look at hours and things like that -- and rough salary -- is this different than something else? And so we decided to go back in time and unfortunately, the ACS and CPS and SIPPs started to collect data in 2008, the 6-question sequence was adopted, by many surveys. HIS, and others. So we don't really have measures for the six questions, hearing, vision, mobility, cognitive, self-care, and independent living, go outside the home type questions. We don't have that data. So we're going to have to use something else. And this is the same period of time -- this is what's called the work limitation variable. And it -- has some issues. People may stop reporting, work limitation because of loss of job. There's not a lot of evidence to that effect. But you can see... If I can get it to go backwards (referring to Figure 2) I'm -- no. What you can see is similar pattern, a decline and then an increase. There was a change in survey design. Such that there was -- such that there's, like, a discontinuity there that is probably attributed to -- to survey effect. We also have -- and this is going to go in the opposite direction: The monthly, initial disability um... Applications per weekday, in thousands. And so we saw a rise in the application rate after the great recession. After really, you know, it started to increase during the great recession, that was my shoulder, (referring to noise made by microphone) a decline shortly after that. Some of this could be attributed to changes in the -- the amount of money given to SSA for adjudicating claims, but in terms of, like, initial application: That would be for an award that would affect that. For initial application this might represent similar to what we saw in the employment rates. And then we have this other measure. And so there was a "Wall Street Journal" article, that Bryce ran across, recently that used these data. And it used the percentage of the labor force, that said that they were not only -- in the labor force due to a disability and for years we never used this variable in part because they make it a distinction between out of -- the labor force, due to disability, due to retirement; and due to school. And then all other reasons. And it's kind of -- it's a coding issue of taking 20-some questions and distilling them down to seven categories. And I've never been confident in the skip patterns. But it does show the similar trend, right? So, again, this is opposite the percentage of people with -- that are reporting on the labor market due to disability, is rising from the great recession, even before, during kind of the heart of the recession, it was increasing. And then it started to taper down, and the reporter noticed this going down. And said okay, I know this variable is collected for a lot longer time period. This is one of the prime labor force questions, it's asked to all participants in the CPS every month, right? And so this is monthly data. Similar to nTIDE. It is monthly. And so I can take this even further back. And, in fact, Bryce and I worked over the summer -- mostly Bryce -- to take this back to 1962. Right? And so this is the first time a lot of the ADA-related research that Rich and I were involved with, went back to 19- -- 1980? Ed Yellen at university of California, San Francisco, did a study that used the HIS back to -- like, 1974, I think. But they have -- what they -- they typically have a lot of discontinuities with regard to survey design but this is the core set of questions, they've only changed the questions twice. Or once, they've only had two versions over time and so what I'm going to do is lay out this period. And so you see the great recession, is the last gray bar. The green is -- the green shading are full employment periods. It is possible to have a recession and have full employment (referring to Figure 5) because recessions are based on economic growth, not employment rate numbers. And it takes a while, there's a lag in employment -- you know, firms start losing money and then they start, you know, dropping employment -- dropping people and then they start potentially reducing wages. So income and employment are lagging. And so it actually -- they're very different data sources as well. And different concepts. And so it is, actually, possible to have them at the same time. And so what you see here (referring to Figure 5) is a steady march-up, and you see the current period is really, very different. Only other period where you may be able to see a decline is the full employment period that... I'm sorry, yeah, the full employment period that started in 2005, '6, there's a slight decline, although -- and these are monthly. And actually that's the dot for the -- that's the dot for the months. That's how tightly these are packed. And so you really do see that the current period is very unprecedented. Right? And so maybe we can learn a lot from this period. And so that's what Bryce and I are going to try to do. There were some changes in 2013. So I wanted to meter this out and make this a slow roll to kind of build the excitement. I know Dave is -- sitting here waiting for this. See what this looks back to years before he was born. Even before I was born. [LAUGHTER] >> So are you guys ready? Here we go! Ah, so there's a big jump-down from that period, that reflects, design. What you see is basically a steady march upwards, there is a kind of a trough in the early -- the early 1980s. There are two recessions -- my recessions graph is not working so well. Because there were recessions in the early 80s. I'll have to fix my recession graph. But right around the 1980s. Which is -- oh, there's not -- they're shown up here, good. You can actually see the regular sessions, the -- the real deep one was 1980. You know, think of the air traffic controllers strike. That's this time and there was a short recession after that. And you really didn't see -- you know, you saw, a decline, during, you know -- there seems to be a decline in the late '70s -- in the '70s, and -- and into almost into the '80s. And you see it declines into the early 80s and rises thereafter. And so, is there something we can learn from the mid... from the mid...uh... The mid.... middle 70s? The other thing I would like to do, as we move forward with this research, is to look at industries, and so -- five minutes? Five minutes! All right. Actually I'm not going to take that long. And so we're going to try to learn from, you know, that decline in the '70s and into the '80s is pretty slight. Right? But maybe there's something to learn to see if similar things happen; so in the '70s, I wanted to take this not as a percentage of the labor force, but as the percentage of the population. That's something that that -- we've looked at employment to population ratios and they pretty much show the same thing for most part of -- for the years we have data. But look at -- as a percent of the population because what was coming in, in 1974 was SSI. You know, so SSI started in '74 and maybe earlier. But... it was '74, right? Yeah. And so something might be happening there. So, again, current time period is really unprecedented. Looking at the full employment rates we've done a lot of initial estimates to see if we can find some effects. I would also like to take initial application rates back to this point. I can only get back to 2000. But folks at Social Security have said that they could certainly estimate that. If I could ever get them to do it. For years earlier. So next steps: For probably the first step is to look at the composition to make sure that -- keep age, race gender, the same over time. Look at SSD applicants and recipiency over time and try to go further back and get the economics of recession and no recovery. One of the things we're tracking with nTIDE for a long time is there was no recovery. We can talk about for the long time we were the bearers of bad news, there was no recovery. People without disabilities were recovering. But we weren't seeing a recovery until the last couple of years, for people with disabilities. So there was perhaps a slow recovery period, and maybe it takes full employment to -- and we -- it was a pretty slow recovery period. Economic growth wise and employment wise, for -- after the great recession. So we also want to look at the -- you know, really the focuses, can we learn anything from the current period? Were there different types of disabilities that were affected than others? What happened to number of hours? Was number of hours also increasing for those who do work? And so that's what we're going to do. In the next few months. All right. I'll stop there. [APPLAUSE] (10:48:08 a.m.). >> Thank you, Nathan. >> There is a laser. They told me there's no laser on this thing! >> You're better than I on that. >> That's forward and back. >> Forward and back. Good morning, everyone my name is Nathan Moon. >> And I'm Fran Harris. >> And we are researchers from the Georgia Institute of Technology. Better known as Georgia Tech, in Atlanta. And today's presentation is titled "contingent employment of individuals with disabilities". And here today we're going to present some of our findings from our first year of research that is drawing upon qualitative, qualitative study we've done today. Just to begin: Simply put, we're becoming a nation of contingent workers. The structure of work has changed over the past several decades. And I would just posit that we all are increasingly becoming a nation of contractors and free-lancers and for some even microentrepreneurs. But you need not need to be a -- a driver for Uber or Lyft to recognize the ways in which our economy is fundamentally changing. So a lot of talk about contingent employment. Well, what is contingent employment? Well, definitions vary. But simply put: It generally contingent employment may be seen as two things -- it is a lack of attachment between the employer and the worker; as well as a reliance upon a conditional factor for employment such as -- you do a job for a fixed limited period of time. Or you have a specific job to be done. So there is a prescriptive literature out there on what contingent employment might mean for people with disabilities. It suggests perhaps contingent employment offers new opportunities to counter the high rates of unemployment or underemployment of people with disabilities. I would push back a little to say, however, that that is -- that actually, the situation is a little more complex. To say the least. On the positive side, contingent employment might allow new opportunities to enter the workforce. Or create opportunities to counter unemployment -- or underemployment, yes. And it may offer flexibility that standard employment cannot offer for people with disabilities. However, there are some drawbacks. There is a potential for lower pay and few benefits. The few studies that do exist most notably a -- a study that was done by the government accountability office, several years ago -- points to the fact that contingent employment is often correlated with lower pay, and fewer benefits for the people who do it. Also, there may not be any provisions to workplace accommodations. It's not clear, what if any, provisions people might get under things such as title 1 of the Americans With Disabilities Act. And finally, contingent employment just simply might undermine the long-held goals of full employment for people with disabilities. And the truth is that we know very little what few studies there are -- would include Lisa Schur's studies. She carried out in the early 2000s, with her colleagues such as Doug, that looked at the reasons for why people with disabilities engage in contingent employment. She threw out three possibility explanations: One was to counter discrimination in obtaining employment. One was to limit SSDI, SSI, to limit your hours so that you would not lose these benefits. And finally: The needs for -- health needs and flexibility that these workers might need. And her studies kind of came to health needs and flexibility as the key drivers. Kennedy and Olney several years later kind of added to that, by -- but by sort of reinforcing, though, the SSDI, was often a major decision, in whether or not people would engage in this so they would not lose these benefits. And then, within the last couple of years, there have been a couple of studies that have come out, that have looked at crowd workers with disabilities, and the participation in the so-called gig economy. Nevertheless major research gap remains, and the actual participation experiences of people with disabilities, in contingent work remains understudied. Particularly we know little, if any about the prevalence or incidence of contingent work among this group; and -- individual and environmental factors associated with efficacious, effective outcomes. Hence the purpose of our project which a field-initiated project funded by NIDILRR, to examine contingent employment of individuals with disabilities. In our project a three-year study we first undertake qualitative research through narrative interviews to identify variables associated with practices and appearance. We also produce and test a survey that operationalizes key concepts and then once we have produced and tested that survey, we intend to establish a preliminary evidence-base of -- on practices. And it's -- I think, it's a slightly different version of the slide deck that we had loaded up. So I'm going to leave it to Fran to take over. >> Frances: What I wanted to talk about was the qualitative research interview. These research interviews are exploratory in nature. We're looking for the complexities of a person's experiences and their choices, in nontraditional employment. These data can generate categories that can, then, be used to build safe -- self-reports. That can be operationalized and used in large populations, to generalize data. For this presentation, we're including approximately 22, 60-minute -- 60-to 90-minute interviews. And we have at least three others; but they're currently being recoded. So they're not included in this analysis. All these interviews were taped and transcribed verbatim. And they were, then -- entered into NVIVO for content analysis. Our findings -- we're presenting our findings as persona. There are so many individuals with unique experiences, that -- in the interest of time, we created these composite characters. Representing the behaviors, needs, and goals and attitudes that were described in people's interviews. Okay the first persona, is David. David is a 45-year-old man, who has a spinal cord injury resulting from a bicycle accident nearly four years ago. His passion is scuba diving. And repairing scuba equipment. Has always supplemented his main income, which was derived from various hotels, and -- entertainment events in which he was a bartender. Since the accident, he's been unable to find full-time employment. However, he continues to work at the dive shop, repairing scuba equipment. He charges a flat rate for a specific repair. And has an informal agreement to invoice the shopowner for a percentage of the fee. Before the accident, his income was approximately $50,000. Currently, he earns $15,000. Together with the SSDI, and his savings, he's been able to comfortably, for the moment anyway, live. However, he is struggling with the numerous health and physicals challenges of having an SCI and having another form of full-time employment. Angela, a 24-year-old female, born with SMA and she earns approximately $28,000 a year. She has always worked in contingent employment. She has a B.A. in English literature and earns approximately $28,000 a year. She free-lances as a copy editor and reviewer for various online periodicals. That are mostly concerned with disability issues. She works on a contract basis, or flat-rate per review. She uses assistive technology, a power wheelchair, leg and wrist braces; dragon dictate, and she also relies on personal assistance from her mom in the mornings and evenings, but she hires a personal attendant when she has to travel between home and one of her jobs. She has recently gotten an internship in a theater that focuses on issues of inclusivity and diversity themes. This internship is to allow her to learn about managing and supporting theater operations. She has a strong social network. She lives with her family. Has a lot of friends, and through these networks, she has gained a lot of relationships with her supervisors, her employers, and her coworkers. And she continues to get work. She also volunteers to edit and review for the same online periodicals and Web sites. And she uses this expanding network, to increase her employment opportunities. Okay. Jackie is a 45 African-American female who has been Deaf since birth. She works as a secret shopper, which is someone who is hired by various companies and organizations, to measure the quality of service; or observe compliance with regulations; or gather specific information about the products and/or services. She's been doing this for many years. As an example, one of the many contracts she has is with a -- is with a drug company. Her job is to periodically audit different doctors' offices in order to ensure that their samples are being dispensed appropriately. She has had lower paying jobs for health benefits. The first she's ever received. However, she used her primary work as the contract jobs, which she enjoys and has relied on for income for most of her working life. Although her income is varied over the years, she currently earns approximately $75,000 a year. Okay. Jimmy is a 56-year-old man, born in Brazil and adopted. He has spent most of his life in Kansas City. He is blind with no light perception since birth. He completed a bachelor's degree in communication. And he held numerous contract-based center jobs, first in debt collections. And then telemarketing. Eventually, he worked for gateway computer and then freedom scientific, all of which were -- part-time, contract work, and done remotely. Based on his computer expertise, and the knowledge -- and his knowledge of accessibility, he would also accept jobs to visit clients with blindness and low vision around the country to help them set up and use screen-reading software. He also worked as a paid musician, part-time, to make ends meet. Occasionally doing call-based work while on the road. Recently, because he was looking to settle down, he accepted a full-time, permanent position as an assistive technology profession and relocated to another state. Okay. >> All right. So those four personas are intended to perhaps give a flavor of the 22 (Nathan) interviews that are represented in the sample, but we could have very well have done any number of them. This slide -- we brought it out in our other thing, but we want to maybe talk to some large trends. One common trend in many of the interviews that we conducted is the role of disability benefits. And people's decisions whether or not to engage in this type of work to give you an example. In one interview for example, a respondent who have been for many years a successful accountant, who had an automobile accident and as a result of the spinal cord injury received, could not -- could no longer work full-time, then went to work as a tax preparer for a tax preparation firm. And as a result of that, you're probably aware since we're in the middle of tax season, that work is seasonal he would work February, March, and April. And doing a lot of for those three months but he would have an agreement with the firm to actually pay him over the 12 months to ensure he never went over his cap and lost benefits. So we would often hear stories like this. >> Fran: Right there was one nurse, I know, who worked part-time. And she -- she was offered a two-week position. She had had a brain tumor. She experienced pain and fatigue and could not hold down a permanent full-time job; however, she was offered a -- a... a contract work for about two weeks. And she was unable to take it, because she said she would be working more, even though she would have enjoyed it, but earning less and would not have met the income caps for her SSDI. >> Nathan: In addition to benefits, flexibility and life-work balance were often a common occurrence. If you remember, David, who is the -- who is our man, who had been a bartender, and then went to work doing scuba equipment repair on a piece rate basis, one of the reasons he can't find a job is need to constantly stretch and change positions from a standing manual wheelchair, and he has not been able to find a job, that is -- is well-suited for his particular health needs. We often -- in two cases we interviewed mothers who had mitochondrial genetic discovers who themselves were raising children with the same disabilities. So while disability might be a reason, there's also simply the matter of being able to rear children and maintain life-work balance. The case of Angela, for example, points to the volunteering, a deliberate strategy that some people have to create their own personal and professional networks for contingent employment. And we recognized, in a lot of our interviews, the differences that existed between people with congenital and acquired disabilities. >> Nate and Frances, I've got to ask you to make your last comment so we can move to the next one. >> Nathan: Sure. With all of that we just simply want to thank you, and if you have any questions, a copy of this presentation, feel free to e-mail either one of us and, of course, our FIP is graciously supported by NIDILRR, through ACL, and we definitely would like to acknowledge our funders as well as our coinvestigators. Maureen linden and Salimah LeForce at Georgia Tech. [APPLAUSE] >> Thank you, next up we've got Megan Henly, who is going to talk to us about job quality now that we've seen numbers on long-term employment trends from Andrew. And we have seen, what I'll call, the human face of employment from some of the qualitative interviews from Nathan and Frances. And looking forward to the interviews -- or about the findings about job quality so Megan. >> Megan: Great thank you I'm excited to be part of this panel. I wanted to share some of the work that I've done with Deb Brucker on understanding the quality of jobs among people with a disability. Before I.... dive into the numbers, though, I wanted to actually start with an acknowledgment. Our work is funded by NIDILRR, the EPM-RRTC, and we have actually worked on three different papers, on the topic of job quality in the last year. All which were supported by this grant. So since I'm -- I have a little bit of time to summarize three papers, just I'm going to keep it very high-level and give you sort of the main findings from each one. Let's see here... I wanted to start by talking about employment more broadly and thinking about what we know about employment as it relates to people with disabilities. So when we think about what it is, we might want to know about employment: Obviously, the first things that come up are the labor force participation rate. The employment to population ratio. These are things that are useful to get a sense of our starting point and what we're beginning with; and I don't think I'm going to be giving away any spoilers even though we have a session next on nTIDE, but I did steal a recent infographic from there that shows, the national trends in disability employment. And if you're not familiar with this, this shows, basically, the changes in the employment rate. Labor force participation rate. For people with disabilities. And while there are changes, from month to month, due to perhaps changes in the economy, what is consistent is that we see a very large difference between people with disabilities. And people without. So others have and we'll talk about these numbers more -- but I really just want to use this as a starting point for framing the results that I'm going to show. This shows -- along the top row, for each of these, we see the numbers for people with disabilities. And the bottom row is those without. So what we see on the right side is the employment to population ratio. For those with a disability, that number is just 30.4%. And for those without it's 74%. So what we really hope to do with this is to add to the research on employment patterns; for people with disabilities, so beyond just looking at this rate, if we look at that 30.4%, those who do have a job, what do we know about the quality of their work? Among those with a disability, who are employed, what is their work like? Are they underemployed? Are their wages comparable to those who don't have a disability? What about the work environment? Are people who work and have a disability just as happy with the work they're doing as those without? So our research question is looking at workers with disabilities. Are they just as likely to have good-quality jobs as those who have no disability? So, again, the numbers that I'm presenting here are focusing just on those who have jobs, the 30% of workers with a disability who do have a job. As I mentioned, I have three different papers, in the works here. Each of those papers, utilizes different data sources. Each one has different measures of disability. And different measures of job quality. So.... What is a good job? That is... very numerous measures, some of which can be pretty subjective. Luckily, we have some existing research that we draw very heavily from. So the European job quality index, uses these different dimensions to assess job quality. So the -- the EU, has researchers with a particular interest in this topic. Much of their work is utilized to be able to compare conditions for workers across different countries in Europe. So we adopt this approach in order to compare across different groups of workers. So in each of our three papers we use different data sources as I mentioned. Each are large-scale nationally represented data sets that have different measures, of job quality available. So the first paper that I'll summarize briefly looks at job quality with respect to pay and benefits. The second paper, I'll summarize talks about the intrinsic quality of work. So this is getting at more subjective assessments of the work that people do on a day-to-day basis. The third paper, includes a measure of benefits, but also has great measures on these remaining three items. Looking at employment quality. Workplace safety. Work-life balance -- something that Nathan and Fran mentioned. So, just to kind of set this up, I'll -- to keep it brief: I'll summarize what job quality measures I looked at for each paper and how people with and without disabilities compare to one another. So our first paper... we focus on pay and benefits. As indicators of a good job, and we modeled this approach on existing research on job quality in the U.S. That used this criteria. So a good-quality job had to meet these three criteria. First, they had to have a job that offered employer-sponsored health insurance benefits. They had to have a job, that paid higher than average wages. And thirdly, they had to have a job that offered some type of retirement contribution program. Further, their job had none of these criteria met they were occurred a bad job. And if they met one or two but not all three we counted those as a fair job, following the work done by Jones and Schmidt. So.... using data from the current population survey: Which uses the six -- the six-question sequence of assessing disability that includes cognition, ambulation, self-care -- hearing, sight, and independent living -- we found differences that were slight. And once we controlled for other characteristics, not statistically significant. So once we controlled for things like gender, race, and most notably, one thing that stuck out to us, which is related to this discussion -- is self-reported health status. So those who said they were -- had poor health were significantly less likely to have good jobs than other workers. But once we include that health measure, disability status, was not a significant predictor of having a good-quality job. Another important measure, obviously, was education: Which would make sense, because the measure of job quality we're using here, is access to benefits and higher pay. Average -- above-average pay, so people with higher education, tend to have higher pay. But that may take away from this paper was after controlling for health disability status was not a significant predictor of job quality. Second paper: Looking at these intrinsic characteristics of work. So these are things like skill. For instance we asked -- the survey we looked at asked is your work related to your highest degree. For autonomy, are you satisfied with the degree of independence you have at work? Similarly, for powerfulness: Are you satisfied with the level of responsibility you have at work? For meaningfulness: About your work's contribution to society. And for self-fulfillment: Are you satisfied with the level of intellectual challenge you have at work? So in order to be counted as having a good-quality job, a worker would have had to say, that they were very satisfied or somewhat satisfied to l five of these measures. And also, I want to highlight here, this particular study, looked just at... workers with a -- at least a college degree. So this is data from the national survey of college graduates. Since in the first paper, obviously, we found that education was a factor in predicting job quality as we might expect. We dug into this paper looking just at college graduates. What we found here, looking even after controlling for characteristics that are related to employment in the literature. Things like gender, race, highest degree -- so everyone had at least a bachelor's degree, but even higher levels of education we controlled for that. Whether or not someone had supervisory responsibilities at work. The field that they worked in, what we found, even after controlling for these, other components, were that college-educated workers, were 27% less likely to have a good job using these criteria. If they had a disability. Now switching to our third paper. This one included six dimensions of employment quality. So we had -- the general social survey had a module on quality of work-life, and so we borrowed from whatever measures they had that were available, which were numerous. So for each of these dimensions, there was one to two indicators that we used to count for employment quality. So for instance, fringe benefits, is actually a qualitative assessment. The worker had to indicate whether or not they agreed with the statement that their fringe benefits were good. For job security we also asked them to assess whether they thought their job security was good, but we also looked to see if they had a permanent or temporary job. And they had to have to agree with both statements, they agreed to job security was good, and that their job was not a temporary job. Similarly, workplace safety. There were statements on that. Work-life balance asked about interference with family life. As well as stress at work. Work support, asked about whether they agreed they were treated with at work. Whether they agreed their supervisor was concerned with their welfare and finally workplace autonomy. Looked at agreement with the statement, that they had freedom to decide how to do their job. So lots of measures here. And in order to count, that they had a good job, they had to have had each of those dimensions, met. So, again, this is GSS data. The disability measure is a bit different on this survey. Just because we're using what's available. So we measured disability as having an activity limitation due to health, three or more days in the last month. So those who met that criteria, are counted here in blue as having a disability. And just 14% of full-time workers, after adjusting for other characteristics, such as age, race, gender, and education -- had a -- what we counted as a good-quality job using this criteria. Compared to 22%, with no disability. Now I do want to mention that most full-time workers, do report that they had job security, good fringe benefits, safe workplaces -- work support. Workplace autonomy -- the real kind of deal-breaker in this measure was the work-life balance dimension. And that ended up bringing the percent with good jobs substantially down. So I -- I want to offer a couple little take-aways from these three papers: First, just thinking about we want to think about employment trends in a variety of ways. If we want to talk about employment, we should really try to dive deeper into the experiences of workers. And think about what their daily work-life is like. I think this relates to what Dave said at the opening and I think that this research actually pairs nicely with what Nathan and Fran showed. But if we're looking at something like job quality, another thing to consider is that there are different ways to assess job quality. And so I think it would be wonderful if we had a single data source that had both a measure of disability and had everything on the job quality index, the European job quality index, if we had a single data source that had all of that that we could use, that would be wonderful. But we're limited, to what we have, and so we've kind of created different papers that looked at each of the dimensions that were available in many large-scale data sources. So our primary findings: So for some of our measures workers with a disability have substantially lower rates of job quality. Than those with no disability. The overall rates varied from paper to paper. As you noticed. But I think what we could -- should consider now is why this might be the case. So I think that some of the qualitative work that was presented earlier is really informative in this -- in this case. It may be that workers with a disability are faced with making trade-offs, where they value some aspect of work-life, over more so than others. So it may be that certain aspects of job quality like work-life balance, as they mentioned -- rank much more higher and though workers with a disability may be more likely to make a trade-off, on some of these other dimensions of job quality that I talked about. So... just something to consider is the fact that different types of people with different types of disabilities also may value certain aspects of job quality more than others as well. That's all I have. Great. (11:18:22 a.m.). [APPLAUSE] >> Thank you very much to our panel. We have -- oh, ten minutes, I was about to say 12! Ten minutes for questions, and answers. I now have some questions, but I want to make sure we get to the audience first. And even before we do that, let me check to see if there are questions online. By chance, is there any questions online? >> Not yet. >> Okay. If you're online and you're listening, this is behavioral nudge, think of a question, and -- and we will get you air time. >> Give you a $100 monopoly bill for the first person who asks a question online. Signed by Dave Wittenburg! >> Dave -- Q. Yeah, so, yeah, this is for Nathan and Frances. I think -- I know the Social Security Administration has had a lot of interest in how, contingent employment has been affecting beneficiaries and applicants et cetera. But I think what you're pointing out is, is -- the potential for disability policy and particularly SSDI program to drive contingent employment, right? That there are -- you gave examples of beneficiaries, who are contingently employed because if they were regularly employed, they would lose their benefits. Is that fair? And -- A. Yes, that's fair. Q. And also, have you thought about that with respect to, you know -- did you come across instances, involving other public policies? And I'm thinking in particular of the Americans with disability act and lots of concerns on evidence at the time when it was first passed that employers were avoiding hiring people with disabilities, because of... you know, accommodation requirements. Or fears of lawsuits if they wind up letting them go. And I have heard from labor economists about, you know, one reason that employers use temp agencies to hire people temporarily, is to basically assess those people to see whether they are people they want to hire permanently and you can imagine, assessing people for invisible disabilities. Or -- might be one thing that's on their minds, but I'm just wondering if you've come across any evidence of that. A. Frances: I think we've come across people who use recruiters and I believe, yes, the recruiters are the first level of assessing whether or not a person is qualified for the job. It is the supervisors, however, who come through the recruitment who decide how well the person can work and it's these people we find -- employees, and supervisors, and employers and coworkers -- often form a very strong relationship, because the work is temp. And as they get a reputation for doing the job well -- they're increasingly rehired and the social network expands which leads to other job opportunities. Q. >> Nathan: At this juncture, we've got all our interviews to draw upon, we're still yet to the point where we would make what I would call generalizable findings, but I -- I can see it flowing the other way as well. I interviewed one respondent who told me that she deliberately chose to work in contingent employment, because she just really did not care for having to request accommodations in the workplace, and that she would -- you know, she had had adversarial relationships with some of her employers because of her -- her disability was a complex genetic condition. She just found it more advantageous for her to simply side-step it and just create her own sort of networks, rather than even though she did for a while, have a permanent job. She left the work. She withdrew from the workforce and they didn't always have to deal with a loss of SSDI in her case. So we see a wide array of experiences and beginning to sort of may be try to understand to go beyond that and get some generalizable findings -- I think might better be able to answer some of your questions. One of the ambitions of our project was to hopefully inform policy, although I will say that -- sometimes I've heard things from interview respondents about -- none of mine have admitted to breaking the law. They all do W-9s, all on time, they've all said so, but at the same time I can't -- I wonder if some of the research might actually result in policies that might run counter to some of the things that we're sort of striving for in terms of employment. Q. Question in the back. Hi, this is Amie Lulinski with the university of Colorado and I have a question for Andrew. With regard to this Social Security applicants' data that you were showing, are you able to pull data, or sus out descriptors such as age, industry, and reason for departure from the workforce? A. Not from that data source. (Andrew) years ago I tried to do that with the CPS, the CPS follows people. And it's been used for some job loss -- actually, Sophy, Deb's colleague has done that. Whether she's done -- merged her disability work into that, I don't know. But not from my data source, but that's an interesting question of -- of reason for exit. Yeah. Q. I have a question from Annette B online. She asks: When talking about contingent employment, and using it as a way of controlling income, so as not to lose benefits, are you talking about losing the SSDI, SSI benefits or losing Medicare or Medicaid? They both have income caps, but the caps are different? >> Frances: Most of the people, identified, SSDI and Social Security, as a benefit they're most concerned about losing. >> Nathan: That's usually where most discussions go to. It has not been typically, so much on Medicare and Medicaid in our interviews. >> Andrew: I've heard Medicaid is bit more difficult. Because it's monthly. Rather than quarterly, and I have friends that are Ph. D. Economists and can't manage their daughter's Medicaid, their older daughter's Medicaid because they keep tripping into it. Just as a question -- while I have the mic for contingent work, a lot of the gig economy requires assets, such as a car for -- you know, Airbnb you have to have a house or a room, for a lot of things you have to have the latest software. High Internet connection, video act. Do you find, in any of your research, that maintaining those levels of -- because you have to have a good car to drive, too. And do you find that assets are an issue? Because that can be one way of supporting contingent work, is to provide asset support through VR or something like that. >> Nathan: I had an ambition when we started this project we would be interviewing Uber drivers and Lyft drivers and I was shocked these are not the sorts of people we had interviewed. That being said. We have purposefully withheld a certain number of interviews because I intended to target recruitment to actually interview these people and ask specifically on these things. So we've held out 5 to 10 interviews we have yet to conduct that we're specifically going to target for these questions, because they are vitally important for us to know answers to. Q. My question was for Andrew? A. Uh-huh. Q. I was wondering what you might think be the causes in the uptake in labor force participation and two things came to mind. One this is a period where automation in many jobs is starting to kick in; and whether there was evidence that may have made things easier. And the other one was how much evidence, was there that this may be on the employer side, that -- so you've been talking about employers being hesitant to -- A. Yeah. Q. Thinking about making the employment decisions, carefully, if they're not wanting to commit necessarily to all the accommodations. If employers are more willing to hire or they're more willing to go further in accommodation because there's, you know, some people would argue we've gotten past full employment, and -- the labor market is -- looks very unique in terms of -- A. Yeah. Q. -- such high demand for labor? A. Yeah, so Dave Wittenburg scribbled down the same question for me, and I don't have any answers yet. I do have some of the same hypotheses to investigate. (Andrew) probably the first thing that I want to look at is local retail, now, I don't have a local measure. And I -- you know, with the -- CPS, we don't have something as low as PUMAs to be able to identify the specific locations, where people are. So that may be limited, but, you know, my sense has always been that it's likely, local, local retail that might be driving it. Because that's where you see -- when you see vacancies that's where you see them. And so to look at the vacancy rates to see if it is -- it is demand that's driving it. Rather than supply. It is still, you know, you have to make a match between employer and employee, and -- I've often thought that -- and the reason I think local is that transportation is viewed as a big barrier, whether they don't have the assets of having a car. Or they don't have necessarily, the ability to drive long distances or have, you know, accommodated transportation. That local would be where it's at; that once I start seeing things local, I'm able to go -- and I also want to look at whether this is just a part-time, is this just part-time? That's rising? It has been drying off, the last few months, have been not as good as -- as previously. So.... I'm not quite sure that would be either. >> All right. I was going to have a question for you guys but I'll ask you. >> Keep going. >> I want to keep on time for the next panel it which is 11:30 and as we've transitioned to that panel, we do not have a break -- is that right? We're just going right into the next panel? What's that? >> Correct. No break. We would like to invite John O'Neill and Elaine Katz to the stage and we'll continue. >> John and Elaine, please make your way to the front. >> As we do this, one of the parting chats I have for the panel is there's a lot of optimistic data coming out of this. We've got increasing employment trends and we've got some pretty creative entrepreneurial stuff going on, with people with disabilities, finding work. I think these are important things, to keep in mind as we move into the third and fourth panels today as we think about these new interventions to support people with disabilities. And kind of what is potentially working about these things? And I just wanted to plant that in the audience's mind as we -- as we first listen to nTIDE, which I think is going to provide even more context on -- on the employment rates. So... [APPLAUSE] thank you for joining us. >> Nathan: Thank you for having us. (11:30:38 a.m.). (Next panel). >> Just start whenever you're ready. I think we're just going to start when -- we're just transitioning without a break. And so, before I say that, Anna, make sure you get the person who asked the question's name, I need to send them $100 monopoly bill. All right so take it away, Elaine! >> Elaine: Okay so we're going to start with a story, an nTIDE story just to be a little bit different. I think Anna, and Andrew -- Megan and has already talked a little bit about it. How did this all start it really started from a reporter, and we're going to name the publication, I think thought a new online publication that started about five years ago and the reporter contacted us about the nTIDE numbers that came out. And he interviewed John O'Neill. Might have interviewed Andrew I can't remember but I know he interviewed John and he just could not get the numbers right! We gave him all this information and he had trouble between the -- labor participation rates, what does that mean? Employment to population rates. And we know it's difficult, that sometimes it goes up, it goes down, but he just couldn't get it right. So from that, we decided that maybe we should start doing the report. And we had talked about it, this was a collaborative relationship between the University of New Hampshire, and Kessler Foundation, funded under NIDILRR stats RRTC so we said, yes, we're going to do this. Ask to make it more interesting, let's put together, not only a monthly data sheet, but let's try to do a graphic. So we had our communication team come up with the graphic. Actually their idea to do this whole report. And we tested it for a couple of months. And we got some feedback. And then it manifested into what it is today and really it's taken five years for people to quote it and look for it. Kessler Foundation does a distribution, UNH does a different distribution. And from the monthly news release, we now have a webinar, and we have social media, and hashtags. And we'll talk a little bit about that in a second. So it's a press release, and an infographic as I mentioned. And it comes out on jobs Friday, which is when the Bureau of Labor Statistics releases its numbers. And the source of the data, is really -- are unofficial unemployment rate for people who don't live in institution between 16 and 64, but really had never broken out disability in the way that we really wanted to break it out. And as Andrew mentioned, nick mentioned it was based on the six criteria of the CPS because it's not easily congested as you saw in the chart earlier and we'll show more in a minute it's year over year. So we're looking the same month from 1998, to -- I'm sorry, to 2018, to 2019. So, again, we now have our Lunch & Learn. If you haven't signed up yet please see me. Please see anybody from UNH and we're happy to add you to our mailing list, so last month we had a really great webinar. It has usually the same format. Starts with John, and Andrew talking about the numbers. Moves into the news, so Denise Rozell is part of our team on our webinars from AUCD, and she provides all the latest government and policy news. And then we get to the most interesting part which is really interviewing from the field, somebody who has been influenced by employment looking for a job, can get their real experience out to our audience. And last month Alex Frazier, from autistic reality, speak about his experiences, as a young man with autism, who's really blossomed through employment and gone out into the world. We had Terrie sue Hartman also speak last month about her work in disability and employment through self communications. And at the end we take lots and lots of questions. So that gives you a brief overview of how it came to be and our process. And now I would like to turn it over to John who will give you more of the nitty-gritty. >> John O'Neill: I'm going to report on the data that we presented last -- last month. For last month. Next slide. >> Actually push the arrow. >> John: Oh, the arrow? All right. >> And Megan already stole our thunder so I don't know who gave her our numbers! >> John O'Neill: Okay. We report on the employment to population ratio. And last month we saw that people with disabilities had a bit of a decline. It was effectively, no change, but there was a bit of a decline, we also report on the employment to population ratio for people without disabilities. And there was a slight increase. In last month. We also report on the labor force participation rate. And last month, of course, you know, the labor force participation rate includes those that are working as well as those that are looking. And there was no change. Last month, and there was a bit of a -- a -- for people with disabilities, and there was a bit of an uptake, amongst, people without disabilities. We also track and Andrew has showed these numbers before, but we track the employment to population, ratio, we track is it since the six disability questions were included in the CPS. And we do that for people without disabilities, as well as folks with disabilities. And from the slide, you can, obviously, see that in the beginning of the great recession, there was quite a pronounced and long--lasting drop in the employment to population ratio for people with disabilities. You can also see there was a drop, also, for people without disabilities. But their recovery happened sooner. And has been sustained across time. And we put a trendline in, so you can -- so we can kind of make sense out of the up and down. And, again, you see the story that I just described. It looks like there may be a bit of leveling off as of the end of last year. (Chart: Future: Intersectionalities). >> Andrew is going to talk about intersectionalities, this is something we're going to add to the nTIDE, possibly, as part of an end-of-year report. >> Andrew: Thank you, John. We have, in the past, done -- year-end reports, years in review; and, you know, it's pretty basic, looking at the average level of month -- these are all, nTIDE is monthly. And we average up all the months of the year, and compare to previous years. And, you know, we're probably continuing to do some work in that area. However, we have always wanted to look at more detailed information. You know, we have the raw data comes out. Not on the first Friday, the raw data comes out usually, it's lagged a week before we get the raw data, where we can actually look at age, race, gender, education level and things like that. We also have another project under the stats RTC, which Sarah is leading, with AUCD. Associations of university centers on excellence on disability. Where Denise is from. And we're going to look at intersectionality, but we're going to do it in context with -- the organizations from the other -- other intersection. So do it with groups that represent those folks. And try to develop an annual infographic each year, with a different group and then continue to produce that infographic each year, as part of the compendium effort. So this is our intersectionality that we're doing as part of nTIDE, so we'll start off with disability, and we'll look at the employment to population -- this is the average monthly for last year. So we took last year's average -- or last year's estimates, and -- this is for ages 16 to 64; so it's roughly the same. Maybe some differences between nTIDE, because nTIDE is pulled off of the Web site from BLS. This is generated from the public use files. So there could be some differences. So this is the average monthly -- this is.... Employment to population ratio. And so we see the gap between people with and without disabilities, and then -- so it's a gap of 43.5 percentage points. There's some rounding errors there. I didn't round at the right time. And then we looked at white only and then everyone else. All right? And so we wanted to see what kind of gap, whether the gap -- whether there was an additional effect of being in those groups. And so what we found for people without disabilities: There was a substantial difference between white-only, and everyone else. And that's 12.1 percentage point's difference. For people without disabilities: And remember, we're starting at a pretty low rate -- low percentage to begin with -- there actually wasn't much difference. Now, this does lump Hispanic populations, Asians, African-Americans, and others into one category. So there could be some variation within that. And indeed, we do. I wish I could bring up the Excel spreadsheet. John said, "Let's do age, race, gender, ethnicity". [LAUGHTER] >> Andrew: Race, by each month, over the last, you know, -- 2008. >> And you said that was splitting hairs. >> Andrew: I did say that and I've split enough hairs in my life. But, you know, when we get to the finite point that may be where it delves over and looks at, you know, what Sarah is doing, with the much larger sample of the American Community Survey, or the data sources she ends up using for the intersectionality with John at AUCD. We did the same exercise.... for gender. And so you see the gap, again, in men and women. And you see the gap between men and women without disabilities. It's 11 percentage point gap. Still pretty big. And for women without -- men and women without disabilities, there is a 4-percentage point difference. But if we look at percentage difference or percentage change between the two, we see they're roughly the same all right? And I'm sure there are a few standard errors around these this would probably not be a statistically significant difference. And so what it is suggesting is that -- disability may not have an -- it has an added effect, but it's the same as without disabilities, potentially, so there may not be a double kind of disability. But, again, I want to check this with the ACS, and I probably want to -- ask Eric to run some numbers with the NHIS, to see if this holds throughout. One of the things we've been doing, and you should see the spreadsheet that John and I and Elaine have been working with -- this thing it's a monster! What we have been doing is looking at the trends in these, to see if that beneficial group -- so the beneficial stuff that John showed, the beneficial jump-up, that John showed, whether that was true for all -- all ages, race, gender, mixes. And look at the mix between gender and -- and race as well. And -- and it appears that many groups have -- many of the subpopulations have benefited -- have had rising employment over the last two years. As compared to previous years. And I forget the -- exactly which ones, but you'll have to stay tuned for nTIDE to see, and -- maybe we'll do one little tidbit per month? Who knows? Who knows how we're going to do this? We thought about having a quarterly report. We also want to do industry -- look at industries and things like that. There's so much we could do. With these data that -- the question ends up being, do we get too far ahead of ourselves? You know, so I didn't provide standard errors or confidence intervals for these, you'll be seeing that more tomorrow. As we bring -- bring in a little bit more careful release of statistics. The -- the key to understand is that many times when you see the slight decline or rise in the unemployment rate, many times that's not statistically significant. And the BLS doesn't really tell us that. So we're trying to bring a little more rigor to the situation. So... With that I'll stop there. And I think we should probably have Denise come up. And talk about our working lunch. One of the things that we had to be very careful with is that the Federal Government and UNH itself is getting more and more specific about spending. And we want to make sure we have a productive working lunch, now, we can't have a keynote speaker which is a typical way of doing it, because we can't bring food into this auditorium, and the people online, and, you know, they don't get the food either. So we don't want to rub it in. [LAUGHTER] >> Andrew: Although, they can participate in this information session that we're going to do during lunch, but I'm going to turn it over to Denise. And Denise, it might be nice to hear a little bit about kind of how you put together the nTIDE news. >> Denise: Sure. >> Andrew: If you could just give us a brief chat -- >> Denise: I am Denise Rozell. I'm with AUCD. I'm the director of policy innovation. And as I say on these calls on the nTIDE calls pretty regularly -- I'm the policy geek. I'm not the research geek. That's these guys, they understand that stuff. >> You say that every time. >> Denise: I do because I have such great respect for the research people but that's not me. Two things: One, what we do in the section on news from the field, that I do on the nTIDEs every month is try to find both what's going on with within federal policy which might be of interest, the new bill that's been introduced raising the wage, for instance, which does away with the 14 (C) program or there's another bill that Senator Casey has just introduced in the last couple of weeks called "transforming competitive integrated employment --" I think it's a very good title with not a good acronym. And we talk about that and that's also phasing out 14 (C) but providing funding for change. A systems change. So what I'm saying is we highlight whatever those pieces of legislation are that might be going on from the -- in the policy arena, that would be of interest to folks on this -- on the call that we do. And then I also try and look for things that have come out in the last month that you may or may not have seen. From a variety of different sources. So it may be a study that's come out. Like some of the ones you're hearing about here. It may be a survey. It may be an international study, you may not have seen. It may be new materials that particularly that the Department of Labor may have either in their GPS side or earned from -- funded by ODEP, or whoever it may be that's out there, something from RSA perhaps -- that's new materials that are out that you may not have seen yet. What I try to keep in mind is that the calls are made up of a whole variety of people. There's the research geeks like me, there's the policy geeks like me, the research geeks like them -- there's families in some cases. Individuals with disabilities who may not fit into any of these categories but, of course, our highly-concerned about what's going on unemployment. I try to pick out a variety of things to put up. And I would encourage anyone who's listening or anyone who is here, if you have something coming out you think this would be a great thing to highlight, send it my way. And I'm just D Rozell@AUCD.org. And I would be happy -- I won't guarantee you, everything that gets sent my way but I will be happy to look at it. That's my piece of it. And I usually do the interview of the person with disability in employment. Sometimes other people interview, but generally I do that interview as well. And I -- again, it's -- the way I explain it to folks in the interview, it's to help people who are listening to the call, who may not have day-to-day impact, by people with disabilities in employment to remember that there are people behind all the numbers. And so we try to get a -- we try to interview a wide variety of people about employment, what they're doing, what's interesting -- barriers to employment. All kinds of things. So that leads me to what we are going to be doing at lunch. What we would like to do is take advantage of those of you who are here, and those who are -- watching on the Web you are more than welcome also to send along your thoughts -- and we're going -- we have a little questionnaire. Which is going to be out there. There's one for each of you. And we would like you to both to fill it in for yourself but also if it's helpful to talk at your table, about what it is you would like to see. So in these nTIDE Lunch & Learns, we're using you as a survey group. So what is your favorite part of the nTIDE monthly webcast, the statistics, the news from the field, the interview with the person with a disability, or the guest speaker -- and then what topics do you want to hear more about? Do you have ideas for a specific speaker? As people on the panels today have been talking, I've been making frantic notes about new ideas. And other ideas to improve what we do in this monthly webcast and, literally, we will pick up your papers afterwards. And we're going to look at everything that you put down. If you have other ideas of things that you think would improve it, that you think you would like to hear about. One of the things we've been doing recently, is a series so, like, right now, we just started the one that you just saw, is the first in a series of three, around employment and the arts, and media. We've done some other series, have some other series planned coming up. Do you like that idea? Do you like the series? Do you have a series you would like to see? I was making notes just now about a series around self-employment, entrepreneurship, contingent employment, all of those things. Might be really interesting. But we want to hear from you. So when you go out there, there's questionnaires. Please fill them out. Please talk at your tables. You may have, depending on how you process information, it's probably pretty clear to anybody who knows me, I process verbally so it helps me to talk at the table. >> I just wanted to add and talk about Denise mentioned we have a series on the media, in the fall, and later in the year we're doing something on -- I think mental health as well. >> Denise: Uh-huh. >> Just as a teaser -- so our next month's surprise guest will be Joey Travolta John Travolta's brother for those of you who do know he runs a program called inclusion films on the West Coast and also film camps during the year, so his program is primarily for adults with autism and other disabilities. Training them behind the camera. So -- and he's great. He's going to be interviewing somebody who works with him, as a cameraman and actually this fall John Travolta his brother hired some of his students to work on the film, and they've -- they've gotten enough credits they've actually been able to join some of the unions in L.A. So it's a really big program; so please tune in. >> Denise: That's another one too: Do you like the fact that we are now doing in some of these cases, the interviewee, is someone who has been through the employment program? Or the -- somehow ties into the topic and the guest speaker? Do you like that? That would be another question we would love to hear from you. So thank you for -- I am going to thank you in advance for all of your great ideas. And hopefully, you'll be hearing more of them soon! (11:53:15 a.m.). >> Andrew: Again, it's a working lunch. What we encourage you to do is to talk to the people at your table. There will be some -- hopefully some people from UNH circulating or sitting at your tables and spying on you to make sure that you're working. We do not want to violate any federal guidelines or anything like that even though it's hard to find those guidelines, if anybody knows where those guidelines may be for the Department of Health and Human Services please let us know. You know, because we've -- it's there's not a lot written down about what is exactly expect PDF of a working lunch. I welcome your input. I recognize if the statistics are not the most important thing you sign for, because Elaine thinks they're not the most popular thing. I think they are. [LAUGHTER] >> We're going to break now, unless anybody has any questions. Any questions, about nTIDE or anything? Sally? Anybody online too? Q. I don't know if this is too late for this kind of question, but we always have difficulty with coding, contingent employment, or casual employment when we calculate an employment rate for example from a study. I'm sure you're facing that and what is -- what do you typically do if a person has contingent employment? How do you code that in terms of, say, the recency of that employment, or the -- A. Yeah. >> Promise? A. Megan might know more about the more detailed questions about contingent work that are in the CPS. For me, I -- I look at hours worked (Andrew) and I look at whether you work or not. I don't actually code up contingent work, but Megan could be -- Megan there might be some questions in some of the supplements to the CPS that could be used. Q. And what time period are you looking if -- A. Yeah, so that's a really good question. So with -- with -- I'm sorry, with... with the CPS usually, it's either the last week or the last month. You know, or the past 30 days. For the March supplement to the CPS it's last calendar year, and the idea, the reason they do it in March is that's when you have your W-2s and all your income information, and so -- it used to be called the income supplement. So you have -- questions about last year, and whether you work full-time, part-time, whether you worked all weeks -- or, you know, -- it has a lot of different questions about weeks worked and hours worked. Q. And no minimum on hours worked? A. You know, I've played around with that. Years ago, where I wanted at least one hour per week, you know, so I've played around with that number. We can talk more about -- about how that's coded. And we should get Megan back in the room, when -- you should ask her, because she looks -- her and Deb do work with the supplements to the CPS, which have a lot more detailed information about this kind of stuff. FROM THE FLOOR: Thank you. >> Andrew: And it was too late for that question. That was last panel, come on! [LAUGHTER] >> Andrew: But we welcome the question. All right. So without further ado, we can break for our working lunch, and, again, chat with people, at your tables and fill out those forms. And statistics are the best ones, so, we already know that. What's the next best one? (11:57:01 a.m.). A lunch break taken... >> Andrew: We will reconvene at 1:15, 1:15 we reconvene. Even for the people online! AFTERNOON SESSION (1:16:26 p.m.) (1:17:08 p.m.) JODY: Are we good? Deb, you want me to go ahead and get star started -- or wait, did you call people in? Okay. (1:17:39 p.m.) JODY: Welcome back from lunch, for those of you who made it back so far. We have a great panel, three speakers talking about opportunities to increase employment among person with disabilities. We'll start with sally Rogers who is from Boston university who will be talking about employment interventions for people with psychiatric disabilities both where things have been and where they are and where they are going. So with that we go ahead and get started. >> Sally: Thank you very much thank you for including me, and it's wonderful to be in this hallowed place. So I'm going to be talking to you from the slightly different perspective, I'm the executive director of the center for psychiatric rehabilitation at Boston university. Where we have been complete immersed with the issue of helping people with mental health conditions either to go back to work or stay at work and that's been a real driver of our energy and focus for the last many years, the Center, has been in existence for 40 years. And I'm going to be talking mostly about this center block of information that has to do with the research and training center funding we've had for the last ten years, that is focused specifically on employment; however, we've been involved in employment issues for people with psychiatric disabilities for as long as we've been in existence, which as I mentioned is 40 years. And as you can see from the slide, we have other projects, going on, that have to do with both promoting education for people. Young adults with mental health issues. Addressing cognitive limitations, among people with psychiatric disabilities. Looking at peer-led interventions, sorry, about that. >> Very powerful mics. >> Raise your hand if I start doing whatever this does. Sally: Dealing with issues around sigma, and involving people with mental health conditions, in the work that we do. And also, trying to scale up and deliver employment evidence-based employment services for people with psychiatric disabilities. So really running the gamut of research, delivery of services, and implementation science. I think I mentioned the Center was funded -- has been funded for 40 years, we were first funded in 1979, and I know that's a little bit of ancient history for a lot of people. But believe it or not, at that time, NIDILRR did not consider psychiatric disability as a real disability. So the predecessor of the Center the founder of the center my predecessor, argued that people with psychiatric disabilities are, like, other people with disabilities: That is, they can benefit from rehabilitation services. They can recover valued roles in the community including that of working. But really, the prevailing message that people received, at that time, and even still unfortunately today, is that you are a person with schizophrenia or unfortunately people say sometimes you're a schizophrenic you will never get better and never work. So forget about it and deal with that reality. But we don't really believe that and we've seen if we take the long view, we have seen lots of changes in the field. Even though we haven't seen a great deal of improvement in the employment rates. But through all this the last 40 years our mission has remained the same and that really is to help people with psychiatric disabilities live in communities of their choice and regain valued role. Chief among them: Employment. So these are some of our guiding principles. We're very focused on conducting rigorous, but innovative research, and also involving people with disabilities and end users in that research. As well as involving stakeholders to disseminate the results of our work and other people's work that will improve the field and move the trajectory down the line a bit. So just a little bit about severe mental illness or mental illness in general. I know we can't poll the people online, but how many people here are -- very much immersed in the mental health world? Are there any people? A few people. >> Person or professional? >> Sally: Either. Both are relevant in the end. So just -- hopefully I won't be repeating too much, but just to lay a little bit of groundwork for how serious of a problem do we have in this country with mental illness, the substance abuse and mental health services administration says that at any one point in time there are between 18 and 24% of the population, that has what they call any mental illness and that can be any diagnosis. From anxiety disorder to some more severe disorders. Such as schizophrenia. And when we look at people who have a severe mental illness it's 4 to 6 population certainly psychiatric issues are something that concerns us all, we know they're a growing part of the Social Security Disability roles and something that we are paying a lot of attention to at the Center including the fact that we have a large and growing number of young adults, in college with mental health conditions and that's something we're beginning to address, because we know the early trajectory for those people in their ability to achieve success in education, will influence their later employment status. And so it's very important to us. But we know that employment rates are still very low. Depends on which statistics you -- you use, and we learned a little bit more about that this morning between 10 and 20%. And, at the same time, what we've seen over these last four years is what I'll call, a -- a paradigm shift, where if you look at the language and discourse 20 or 30 years ago it would be all about stabilization and maintenance, we can get people out of mental institutions, and maintain them in the community; very little about that people going to work. But that's changed. Even though, we're not having as much success as we like, definitely the -- the paradigm has shifted. If we go back a ways, actually, when I was a vocational rehabilitation counselor, in the state of New Jersey, I hate to admit it, but I am from New Jersey. The only thing that we could offer people with psychiatric disabilities was a sheltered workshop. And I remember an attorney coming to me for vocational services and he really wanted to return to the world of work as an attorney but in a different way at a different level of responsibility and work-life balance. And all I could offer him was you can go to a sheltered workshop. That's a place where people get together during it day and they put together widgets or some kind of small manufacturing assembly type of job. And that was really all we had to offer. Things have changed dramatically in the last 30 years, thank goodness. So that was, what we called, a training place, we trained people and then we get them out into a real job. And what we've found is people would languish in sheltered workshops for years and decades, and some people would never get out, to real work to competitive employment. So things have been shifting, more to, what we call, a place in training model, a model where we get people out into the community, we get them into a job and we wrap supports around them. The other major driver of changes, I think in our field, is the peer movement. So in the mental health world this is the idea that a person with a lived experience, of a mental health condition, is in the best place to help another person with such a condition. Really recover valued roles and recover meaningful lives. And so the peer movement has changed what we do in terms of employment services. And also has been a pathway out of unemployment for people with mental health conditions, a huge pathway and that's been a wonderful thing. But probably -- one of the major differences with mental health conditions and psychiatric disabilities is that in -- every state in this country there is a mental health system. And so we have a way that we can influence people's employment status, through the services that we offer through these departments of mental health. Not every disability has that advantage. And so because of that, and with that, what grew up was something called supported employment. And I would say starting in the '80s, supported employment was originally very generic term. And now it is an evidence-based practice. The supported employment model was developed by drake and his colleagues at Dartmouth, now at Westat, now have 20 randomized clinical trials, all suggesting that if you provide supported employment, you will have better outcomes than if you do not. So every -- every study, experimental group is better than the control group of services as usual. It's also been studied internationally, with -- some success although a little bit less than in the United States. We've also found that not only can it help people get employed; but it can also help reduce mental health costs. Because we know that employment does have a lot of benefits in terms of mental health condition. These are just some of the IPS principles. I won't go through those, they are online if anyone is interested. But in terms of shifting the paradigm: It really -- we're going to take people's interests into account and we are going to focus only on competitive employment. No more this sheltered workshop. No more volunteer work. No more progressive employment. People have to get into competitive jobs that is our primary focus. The problem with all these 20 -- or so randomized trials is that from what we look at, in terms of the data, people are not getting what we would call decent work; that's I think the World Health Organization, definition of decent work which very much coincides with the quality of work indicators that were mentioned earlier. So people are almost exclusively in dead end jobs and no health benefits, retirement benefits or anything like that. And one of the largest studies done that speaks to us, in terms of the work that still has to be done -- is the mental health treatment study. Conducted by Westat and others, looking at individuals who have a psychiatric condition and are on SSDI, inviting those individuals to come into a study and to be randomly assigned to get supported employment and expert psychopharmcology or medical treatment, or -- or no services at all -- just staying as a control participant. Only 14% of the people who were invited to come into that study actually did come in, so there are questions about how comparable they are to the larger group. And we did find or the researchers found -- that individuals were more likely to engage in competitive employment, and they worked longer; so those are encouraging findings. They also experienced less mental health treatment in the experimental group; so those who got supported employment, worked longer, and maintained competitive employment for more months over the follow up period. But the problem was despite these findings, few participants almost none really -- worked long enough to exit Social Security Disability. So no one was able to work at a level or a quality of work that would enable them to leave the disability roles. And when we actually look at the cost-benefit, for the person, essentially people were making in the experimental group over the two-year follow up period were making about $11 more per week. As a result of getting the services which an economist said basically, is -- meaningful amount, but not one large enough to make a real difference in the lives of these vulnerable people. So that is pretty much where we are, we've tried many things to move the needle but in terms of really getting people to decent work, to jobs with -- that have sense of quality and.... permanence, we are really not there at all. The other problem we were having in terms of looking at the results of these randomized trials is, what seems to me to be a big issue about how we measure employment success. So many of these randomized trials will report results of an employment rate of 50, 60, or 70%, but that usually -- represents only one day of employment. So over a two-year period if a person can get a job for one day they're counted in the successful employment rate. And I can tell you from experience, that pretty much anyone with a mental health disability can get a job for day. So having meaningful indicators of employment success, is so critical, for us, and -- and many, many other disability researchers and groups of people advocating for people with disabilities. So let me tell you a little bit about that, setting the stage for the Center, really wanting to -- augment what supported employment does. So we have established evidence-based practice. It's out there in many states but we know it's not sufficient, it's not where we want to end our discussion about what we need to do to improve the employment lives of people with disabilities. So we've conducted several randomized trials, to fill these gaps and what we see, in -- in individual placement, supported employment. And our goal is really, to test these new augmentations, to test various both cognitive interventions, peer-led interventions as well as other interventions. Also to do some secondary analyses and policy studies of the adoption of employment services for people with mental illness. So I just want to give you -- we only have two minutes! Yikes! I'm just going to give you a little bit of a smattering of the kinds of things we're looking at. We are not only interested in, you know, skills and employment rates but some of the softer kinds of things that we know affect people's employment success like motivation to work worker identity things like that. We've built various initiatives. This is a list of those initiatives focusing on coping at work. Empowering people to feel like they can be a successful worker. Using peer support specialists to support people while they're at work and in their employment journey. And the last one. Opening doors is a career guidance intervention designed to fill the gap between supported employment, which focuses on lower level jobs; and people who have a mental health condition, but also have career aspirations. So that's our goal with those projects. Let me skip to... the conclusions. So when we take the long view, significant strides have been made in employment services from offering simple sheltered workshops to focusing on mainstream integrated and competitive employment. But the services that we have and that are accessible, are uneven throughout the country, and even within states, you might be able to get supported employment in one part of the state but not in another part of the state. And most importantly: Individuals who engage in evidence-based supported employment do not achieve decent work. So I suggest almost no one goes off of disability despite getting the absolute best of supported employment you might find in a randomized trial. And we continue to believe new interventions and innovations are needed, to be developed and tested to try to advance employment services, make them more accessible; and really promote people on the continuum or the trajectory of vocational recovery. Acknowledgments: Funding from the national institute on disability independent living and rehabilitation research, who has been, a great partner, and I think, with all the people at the Center has really changed the thought process, the discourse, the paradigm, around psychiatric disability and we're grateful for that. Thank you very much. [APPLAUSE] (1:34:41 p.m.). JODY: Next up we'll hear from my colleague Yonatan Ben-Shalom who has the fun task of describing four studies in 15 minutes. [LAUGHTER] >> Yonatan: Am I supposed to bring this up? Oh! Okay. Good afternoon everyone. I'm going to be talking about special section that we published recently in the journal of occupational rehabilitation. Called opportunities for early intervention to avoid prolonged work disability. I do want to start by thanking NIDILRR, for funding both the organizing of a special section and two of the studies, and thanks to the Social Security Administration's disability research consortium for funding two of the other studies. So job loss after a disability, this slide here, and that means a new or deteriorating medical condition that increases the risk of job loss and exit from the locker force, and this often leads to reliance on public programs. Including Social Security Disability insurance, and Medicare. For the person that means reduced income. Without health insurance and social isolation. From a federal perspective, that means strain on SSDI and Medicare trust funds. So what is early intervention? In general, we're speaking about programs that are designed to help soon after medical condition develops. While people are still attached to the world of work. The idea is that, these programs improve their well-being; make them more likely to stay in the labor force and less likely to apply for SSDI. And there's all sorts of evidence-based interventions that fall into this bucket of early intervention. They can be workplace-based; they can focus on certain conditions such as low-back pain and mental illness. And they can be multidisciplinary and multisectorial. So this slide shows you, the net benefits of successful retention of people with medical conditions, this is from a study we conducted a few years ago for the Department of Labor. And what you see here are the net benefits... from few different perspectives, the state, the Federal Government, and the employee. The main take-away here is that, the Federal Government has much more to benefit than the state. Given how much benefits come out of the Federal Government. Population, but the biggest winner from staying at work, is the employee. And just to put a fine point on this, this is for somebody who's -- received disability onset at age 50 and they're calculated -- these benefits are calculated through retirement age. So it's clear that the many perspectives early intervention and helping somebody stay in the labor force is beneficial. But there are many barriers, financial incentives, workers out of the labor force and into SSDI, there is no public agency that has clear responsibility to help people who are working, and then experience a medical condition, before they get on benefits. The existing public and private support that do -- that are around are really fragmented. And from that perspective, and from the perspective of a cost-benefit analysis that I shared earlier, it's pretty clear that federal leadership and funding is necessary to encourage a state to help this population. And when you think about how to help this population, one of the main issues, to tackle first is how to effectively identify these workers. There's no automated system in place, you know, that tells you that this person, is at risk of prolonged disability and now is the time to swoop in with early intervention. So, the evidence shows the window of opportunity closes around 12 weeks after onset of a work disability. Delay intervention beyond that may prolong the absence for work and ultimately increase the chances of job loss and workforce exit. But efficient targeting is also important. There are some people who may retain jobs even without intervention. And all those may not benefit from the intervention so it's key not to provide services to people for which they won't make a difference, really. So we get -- this brings us to the special section four articles that focus on options for early intervention, three of them examine options for using disability claims data to identify workers who could benefit from early intervention. So these studies are quantitative. And then we have a qualitative study that summarizes, employer perspectives, about retaining workers after they experience a disability. The section is readily online and in the December issue of the journal. So I'll start with key findings and then I'll go to show you -- illustrate the findings, how we... they're shown in each of the papers. We find out information and initial claims can help identify long-term claimants. However, we find there's a trade-off between giving workers services immediately and accurately identifying the ones who will even benefit from them. We find that long-term claimants at least in one state are similar to SSDI, nationwide. And from the qualitative study we find that employers do not necessarily deploy resources and processes consistently across cases. So let's talk about the first main finding. Information initial claims can help identify long-term claimants. We found these in three different settings. We looked at Rhode Island, temporary disability insurance program. California's workers' compensation and state disability insurance programs. And in database of private employer-based short-term disability insurance programs. And then the important factors here are not surprising. There are age diagnosis, industry, physician, specialty and so on -- the important point is certain databases do include this information that can be used for this sort of risk assessment. And then we also found trade-off between immediate services and the accuracy of targeting. So many short-term claims resolve quickly. And so waiting for that attrition can substantially increase the efficiency of the targeting. We found that both in the California data and in the private insurer data, um... however, it was not as targeted as we would want it to be, certainly room for improvement. So collecting more information, soon after the claim could improve the efficiency and timing of the interventions. So this is a -- a chart based on the private short-term disability insurance claims, just to illustrate, this trade-off. So the bars show the number of claims that are left after zero, two, 4, and six weeks after the initial claim. And you can see that by week 6, we're left with less than half of the initial claims. So more than half of the claims resolved simply by waiting. Some of them might have benefited from services that could have helped them return to work sooner, but in general, waiting for six weeks can really decrease the number of interventions you would be considering. The line shows, the number of remaining claims that would have exhausted their benefits in this case 26 weeks and you can see the more you wait. The riskier -- the pool of claims that remains, to target. In week 6, we can say that 7.7% were going to exhaust their benefits by week 26, if you wait until week 6, that percentage goes up to 70%. So the general risk of the people there, to exhaust their benefits is higher. We also find the California study that long-term claimants are similar to those nationwide we were not able to match the data what we did is look at the statistics, the distribution of conditions, for state disability insurance claimants and workers' compensation claimants; that lasted for 12 months. So these are long-term SDI and workers' compensation claimants. And I won't go through the whole list here, but if you look at the very two top groups of bars, the very top one is musculoskeletal system and connective tissue and the second group is mental disorders, you can see at least for state disability insurance claimants, they had very similar rates of those conditions, as did those nationwide. In the qualitative study we found employers are inconsistent in their deployment of resources and processes. We interviewed HR professionals, who work for 14 Arkansas based employers 50 cases in which an employee developed or disclosed disability. The reason we focused on Arkansas is because Arkansas VR agency actually has a program that services -- provides services to employers who want their help. And providing early intervention services to employees. In the interviews, we found factors that influence employer's efforts to accommodate and retain workers, included -- include their access to public resources. Their own resources. Their ability to communicate, with the worker and others. The workers tenure and performance. The active sedentary nature of their job; and the nature of the worker's health problems. So the main point here is to understand that there are a lot of factors that go into the employer's decision, whether to accommodate or not. Obviously, there is the ADA, and requirements there. But there is still a lot of variation in what employers actually do, given what is presented to them, when -- when they're considering a -- a -- a specific individual. So the -- the key take-aways here, there is a strong opportunity to help workers with medical conditions stay in the labor force, there's a lot of strong evidence base around that. There's an opportunity to build on existing state and private systems to identify workers during the critical period; there's also a couple of important considerations that come out of this research. One is the question of how to improve the efficiency of targeting. I think we've shown a bit about how we could go about that; that -- if you really want to target efficiently, you probably want to have some -- some additional screening process to make that work better. And then the question of how to reach workers that are often overlooked by employers? I think when we think about policy design for early intervention, we need to realize, that employers have their own considerations, and support to consider how the policy, interacts with the employer's perspective. And finally, I want to end with a question of will retain show the way? Some of you may not be familiar with Retain. Retain is -- retaining employment and talent after injury or illness network. This is a collaborative effort led by the U.S. Department of Labor and the Social Security Administration. And in the first phase, it provides eight state grantees funds to develop and test early intervention approaches. So this is really, a great opportunity to test these ideas, on a -- on a -- in states in a practical way and modeled after promising programs operated in Washington states, workers' compensation system. And this will lead to -- Dave Stapleton's -- presentation that he'll begin in a minute. Thank you. (1:47:33 p.m.). [APPLAUSE] JODY: Dave is going to talk about something he and Yonatan have been working on about a promising service system for early intervention. >> Dave: Again? >> Patience. >> Dave: So while we're waiting. -- there we go. Some of you are probable wondering under my name tag, it says tree house. So I used to work in Mathematica, but when you retire they put you in the tree house. [LAUGHTER] >> Dave: Actually, I left Mathematica, in the November and -- with the intent of working as an independent consultant for a while before I really retire, and my wife suggested we take the LLC Tree House Economics, nobody would take us seriously; and so far, nobody has. [LAUGHTER] >> Dave: I'm going to report on something called the employment eligibility service system. You may remember in 2015, there was all kinds of concern over the SSDI trust fund, which was expected to run out of money in 2016. And there was a big effort to develop policies, that would save the trust fund, save the SSDI program from doing mandatory cuts across the board. And, I was very much involved in that effort, SSDI solution initiative solicited policy proposals. Yonatan Ben-Shalom, put in one of those and it got a lot of attention, a lot of interest. Now the first chapter of proposals in the book they put out in 2016. And -- early in 2018, we were asked to consider how we could actually take steps towards implementing our proposal. So we've been working on that, some with the help from NIDILRR and some with the help of the centers for responsible funding. And measurement -- RRTC with New Hampshire. I'm going to give a reprise about what this proposal is about, but mostly report on thinking about how we could move this forward which is related to what Yonatan just talked about. So there's my coconspirators, Yonatan, and the support and usual disclaimer applies, I'm not going to go through that. I think you all know what it is. Before I actually -- actually present this to you, I want to really make it clear that our intent here is to develop -- have a change in policy, a change, really major change in policy and programs in disability that produces better outcomes for workers, when they have a significant medical problem. All right so they get faster assistance than they can get now and experience less economic loss, less social isolation because they're more likely to be able to stay in the labor force, rather than become dependent upon Social Security Disability benefits. We also want to seek to modernize the SSDI disability determination process, following same process that was first established in the 1950s. And we also think this has a potential for greatly lowering SSDI and Medicare costs although that's yet to be realized, of course. And we also think there would be better outcomes for employers, and I'm saying that because I think some people when they first see this -- oh, you're trying to take away, the disability benefits that employer -- workers are entitled to, which is not at all our intent. However, I will have to say that, you know, if you don't do this carefully that could be an effect. So this is a caricature of what the current system looks like. A person who experiences a back problem, in the picture. When they have to stop working because of an issue, a medical issue. If you can't see very well, don't worry. It's a jumble and there's not much to see here, that's part of the point. It's a highly fragmented system, and workers need to navigate it pretty much on their own. Although it depends very much on what their circumstances are, they can, though, apply for Social Security Disability insurance benefits and if they're lucky they'll get them as soon as five months. That rarely happens. They can also try to, you know, through providers healthcare providers, but maybe if they're covered by workers' compensation, or -- or private disability insurance they can get help there. So there's very much they can do, many people try to do that first thing, and apply for Social Security Disability benefits anyway, the point is the system is badly fragmented. Incentives are misaligned; and we have a lot of workers who experience, what we call, unintended or preventable long-term work disability. And become eligible for SSDI, and we also know the -- for decades, the disability determination process for Social Security has been performed quite poorly in a number of different ways. So that's what motivates us. Also on the slide, I want to point out that there's different institutional structure, the Social Security Disability program is all federal program; and that's overlaying a -- a support system for employment supports, healthcare, et cetera, which is all run by the private sector or states, right? So that's a really major institutional issue when it comes to making any changes like the one that we're suggesting. So here's -- here's our stylized picture of what we're trying to aim for. And this is a system that's integrated, where employment supports are integrated with a disability determination process, where somebody who initially experiences job loss gets help right away. Gets evidence-based support to try to get back to work. If that doesn't work, they can -- they can go on to Social Security Disability insurance. But -- but you go along a path, it gets you the -- return to work supports first. And the only -- only when those don't work out, or when it becomes evident they're not going to work you're actually getting the SSDI, again, I don't want to get into details. We'll see a little bit more of this as we go on. I guess a feature which is right in the middle, is that when you get into the system, the EES system as we call it, and you can think of that as DDS2.0 for those of you who know disability determination services there's a decision about whether you're eligible for benefits at all. Maybe there's a significant condition which is not going to last very long anyway, in which you don't get anything. But some people will get directly in to SSDI, let's be clear they have such a significant condition they're not going to be going back to work in 12 months and may even be expected to die given the SSDI quickly. But everybody else get on this return to work track. Where they get some -- some temporary support, both cash support as well as services to help them get back to work. And I'll tell you a little bit more about those in a minute. Major challenges to building something like this. I mentioned the institutional difference between Social Security programs, which the is a federal program and the state and private avenues that are involved that a working might be able to get support from. And along with that there's a legal, regulatory, administrative, and service infrastructure complex built around the Social Security Disability insurance program as well as other programs that we have. Many stakeholders how do you change that massive system? And, you know, that seems like a really Herculean task. How can you possibly do that? And also, high potential -- I mentioned before -- potential for harm, that -- unintended consequences if you try to make big changes in a system like that. Clearly going to take years to make changes like this. You can't do it overnight. Can't develop systems. And, of course, trying to make that in a very unstable environment. So I think I can just quit there and go retire and leave it at that. But we're really challenged to say how can you do it in a system like that? And I think you've got to think about the problems we've had with the current system and the decades and decades of work to people try to nudge the employment rate up and reduce the number who are on SSDI and it never seems to get better. It's a Sisyphean task, where we're pushing a rock up the hill and we keep pushing it and it's a lot of work but we never get everything, we have a choice between a Herculean task and a Sisyphean task, and I want to show you how we can make the Herculean task maybe manageable. Not easy but manageable. So here's our development approach, and I'll get my water and continue. [LAUGHTER] So we want to build on this effort -- Yonatan talked about the retained grants. And we've used the term workforce retention service, or WRS system, to describe a generic program that's really designed to help workers when they immediately lose their jobs because of a medical issue. Get back to work quickly. Within three, five, six months. Okay? So we want to build on that, as a first phase of developing an EET system and then go from there to expand on it and I'll come back, and talk to you about that. Yeah, so.... so this is -- a simplified version of what I showed you earlier for what an EES system would look like and it assumes we have a WRS system in place, one of these states has actually created, a system under Retain, and it's working well. Up and functioning, and what we want to do is build the rest of the EES around it. So the worker may get maybe six months of help from the WRS. Workforce retention system. But not succeed at getting back to work, in fact, during that process, it may become very evident they're not going to get back to work within six months. And at that point, they transition to the EES. And, again, it goes through the triage process like I described earlier. But we've now got a very reduced set of workers who are going there. And most of them will actually be eligible for services. Very few will be -- refuse services because they've already been through the WRS and been out of work probably for five or six months already. And but -- so then we -- again, fast-track some of them on SSDI at that point. But we focus on those who have a -- still have a shot of getting back to work within the next six or seven months okay? And -- so that's the initial triage. And now, we've got to implement decisions that have been made -- in that initial stage. And SSA is going to pay benefits just like it does now for SSDI, for SSDI beneficiaries. But then we have the employment supports benefits. So the EES, itself, would have counselors who are skilled in helping the individual workers develop a return-to-work plan; figure out what they need to go back to work over six months or so, and make sure they can get those supports from the local environment. Provide some financing for them, but get financing from other sources. And in the meantime SSA is going to be paying a time-limited cash benefit. So it's, like, a temporary Social Security Disability insurance benefit. Basically the same thing, right? And it's going to replace SSDI for that six or seven months. Hopefully, the person will get back to work back then and won't ever have to go on SSDI, but they might. They might. So we've... we've built in benefits and protections for workers, who are eligible for employment supports. So the immediate cash support, are you going to get this five, six months after you've stopped working? Which is as soon as you can possibly get SSDI now, you're going to get help to stay at work. And then you have a right to redetermination of SSDI eligibility if your efforts to going back to work are not succeeding for whatever reason. And in addition, now, the Social Security Administration has the opportunity to see that, yes, you have a medical condition; and, you know -- it wasn't significant enough for us to give you disability benefits before. But we can also see if there's support available to you to get back to work. You're just in an impossible situation. Now we're going to give you benefits, okay? So it's a different -- different set of rules that applies in the current law but with the same purpose in mind. You have the right to appeal your decisions, EES made a decision you don't like, then you have a right to appeal that just like you would have a right to appeal denial of disability benefits now. So that's it, in a nutshell. And, you know, there's a lot more to it than that. And our paper is not out yet. It will be out I think by the end of the month. We're shooting for that. Although I would have said the same thing at this time last month. [LAUGHTER] >> It would have been January, I think. So -- and I think, you know, what we've found, we've had a lot of people reviewing initial draft of this paper -- there is this real broad support for the idea of an EES, for integrating employment supports into the process of applying for disability benefits. You make sure workers get a fair shot of giving back to the labor force, before they get into.... SSDI, but also to change the way that we do disability determinations. So that we can take into account, of not just sort of theoretical supports that might be out there; able to help the worker, but actually what is out there? And what they're able to access, where they live, right? We've also encountered a great deal of skepticism about the feasibility of change. We are skeptical ourselves. But we think it's too early to give up. And so the paper will be out there. And we would encourage you -- it is out there to read it. And hopefully, we'll see what happens after that. Thank you. (2:02:10 p.m.). JODY: I think we have something, like, ten minutes for questions, I don't know if there's any questions from online yet. >> Not yet. JODY: Not yet. All right. I'll start if there's nobody. Is that the goal -- >> Audience. >> There's no --. [LAUGHTER] Q. Hi, Daniel Davis with the ACL. One of the questions that I have is how some of these inter- -- interventions might interact with -- with the recent um... advent of Medicaid work requirements; and um... Is there -- is the list of the states participating in the DOL program public at this point? Is the second part of that. A. Yeah, the list is public, there's a page on the Office of Disability employment policy (Yonatan) agency at DOL that shows each of the states and has a short description of their program. With regards to work requirements: We haven't really thought about it much in this context. I mean, at least in terms of early intervention to people who are -- these target populations are already working. And most likely not on Medicaid. So I think at least intersection is quite small. Obviously, if they lose their job they might end up on Medicaid. And that's after losing their job, so that might -- it will be harder for them to return to work, but the -- the main idea of Retain, is to help people never leave the workforce. FROM THE FLOOR: Q. This is Hugh, for David. My question is, for the EES, you mentioned that a consideration would be the availability of resources, and services and supports. In making decisions for people. And I wondered if the resources would include the availability of jobs in particular, with -- with issues, like, geography, and location being a factor in the decision-making. A. David: Interesting question, not the question I was expecting but I'm glad to see you, I was told you weren't going to show up today. So this is good. [LAUGHTER] A. We -- we have thought about that issue, and I was thinking about the Medicaid issue, as you were asking your question because the Medicaid situation is relevant for the resources being available that you might need, because a lot of people do need healthcare, right? So but, yeah, so... one of the big issues with the current disability insurance program is that whenever we have a major recession like the one we had in 2007 or even less severe ones a lot of people apply and some of them get benefits. So at least people have been working, obviously, but they've lost their job, had a medical condition, and qualified for SSDI. And, you know, if we're in a major recession like that there are going to be people coming in the door to the EES, who don't have a chance at getting back to work within six months, right? So... I -- our thought is that the way we've designed this thing, there would be an opportunity during a recession, for the Agency, whoever is responsible for the program, and it would have to be written into Policy, to expand the period. Length on the period but you're giving time limited benefits but the time limit is longer. And the advantage of that, relative to current Policy, is it tells somebody who is going into SSDI, first of all, here's assistance for you, right? So you're not -- and we know you're going to have a harder time than some people getting a job, when the economy recovers. We give you longer time to do it, but we're not expecting you to stay on SSDI forever. So I think that's -- that's a possible way this could go. It's not something we've developed a lot in detail, but I think it is an option. It's a good question. FROM THE FLOOR: Q. This question is from online. From Annette B. Will the EES, emphasize things like technology and peer support? Or will it stick with just medical supports? A. (David (laughing) our intent is whatever works. I think if it's technology, or -- individual -- whatever works is -- is our intent. FROM THE FLOOR: Q. Dave -- A. Yes. Q. -- I have a question, John O'Neill. This is kind of futuristic, but if the ultimate demonstration of -- around SSDI, ever becomes reality -- how would this program fit into that context? A. Dave: Oh, you mean the ultimate demonstration? Q. The ultimate demonstration, I'm sorry. A. Do you want to explain what that is. >> John O'Neill: That would be, if I understand it correctly, SSA, is proposing a demonstration, where people receiving SSDI would be allowed to work, as much as they want. Earn as much as they want; and not have their SSDI benefits taken away; is that correct? A. Dave: Yeah, yeah, and I think the intent of that idea is, just to see how much beneficiaries could work, if they weren't -- if there was no disincentive to work, right? And because I think right now we don't know. You know, we -- there's a national demonstration -- increase in earnings and employment was tiny. And we don't know whether that's because people just can't work more than the amount, or whether it's because of -- it was still a significant disincentive, you know, basically a 50% tax rate for earnings above the SGA level. So I think that's the intent. I hadn't really thought about it in connection with the EES, because EES is really targeting people who aren't in the SSDI -- already, and -- you know, I don't think it's the intent Social Security, certainly not their intent to change, you know, -- change the benefits so that it's the ultimate benefit; but I -- I think that the -- one thing that I do think the EES would really help with is, tailoring long-term benefits. So not just temporary benefits, but long-term benefits to the needs of the individual, right? You observe somebody trying to go back to work for six months. You can see they're never going to be able to supports themselves at a decent level through work alone. But maybe you could give them a partial benefit? And just fix an amount and they get paid that and they work as much as they're able to. So... that probably doesn't answer your questions, but it was a difficult question. [LAUGHTER] FROM THE FLOOR: Q. Hi. Ken. I have a two-part question, but related. It was discussed, that employers go through, a process, determining, considering many factors if you're going to -- if I understood correctly -- if they're going to offer accommodations to an employee; and I was wondering if you could discuss, what possibly, from a public policy perspective, might help the employer make that decision to offer that accommodation. And also, in EES, when you offer the supports, and you get that employee that has a disability, the potential employee who has a disability, ready for employment, if those employers aren't willing to offer accommodations, to enable them to do their job, then what happens? A. Yonatan: I'll try to answer the first one, based on the qualitative paper, in Arkansas. And, you know, maybe accommodation is too specific of a term, but it's more generally how much effort they put into, you know, trying to have the worker, keep their job or retaining them at work. In terms of a public policy, you know, -- so Arkansas is one of the very few states who has a public program through the VR system, where they offer a service to help the employer figure out how they can implement some best practices and keep the employee. They serve maybe 200 people a year. You know, they would love to serve more, and probably don't have the capacity. Many employers who could use their services are not aware of them. So I think what one policy implication is to -- have such a service in more states and have more publicity around it. And make sure that employers know that it's -- that it exists. In terms of... You know, thinking about how to account for employer variation in how they would treat employees in larger public policies -- such as Retain, EES, I don't know that I have a solution to that, but I -- I think it's important to recognize that, there are a lot of factors that employers think about. And -- you know, we -- we can keep saying that the ADA requires them to provide reasonable accommodations. But the facts on the ground are that, if somebody just started working two months ago, they will put much less effort in that to keep that person than they would, somebody who worked for two years. Or if they deem that worker to be a problem worker, they will put much less effort than they would in another employee. If they have a mental health issue, that they think is too complicated to deal with, they will put less effort than they would, where there's a clear path forward. So I don't have the solutions. But I think that's -- these are all factors that we need to consider, and in the design of early interventions. >> Can I ask a question? Is that allowed. Q. I think -- in the mental health world a lot of people don't want to disclose they have a disability (Sally) so they would be looking more for what we would generally call in the -- in the labor force, flexible workplace supports. And we've found that those are associated with people being able to retain their jobs, but it doesn't necessarily require disclosure and a formal accommodation. How would that factor into retain? Would a person have to disclose a disability to be eligible for those employment supports? To their employer? Because that's a huge barrier for a lot of people? A. Are you asking about Retain or EES. >> Sally: I'm not sure. Retain, I guess. [LAUGHTER] >> Yonatan: Well, I think -- in Retain, there's -- you know, you can see how the -- the states describe their programs on the Web site. And most of them, by design, of the grant announcement, have a focus on musculoskeletal issues. But there are states that also have this additional focus on mental health. In general, the disclosure here would not be through the employer; it would be through the medical system. The employers are sort of not -- not the funnel to -- for the target population, but it's more the medical system. >> Sally. But the employers give the flexible supports or the accommodations? >> Yonatan: Well, so some states differ in how -- they're still developing their interventions and you can't really -- I'm not in a place to say what their plans are in that. I think it's something they're still working on figuring out what's the best approach, but it's a -- the issue of what to disclose to the employer, is definitely, first of all, would never be done without consent from the worker. But the question of how much of that to do is -- you know, is one of the things, the states are figuring out. >> Dave: I think it's fair to say it's clear the intent of the Department of Labor to gain the cooperation of employers, when that's possible to do. And that that can be an important part of the return to work process; but I don't think, you know, nobody is going to force somebody to disclose information to an employer that they don't want to disclose. >> Yonatan: It could be an educational component that doesn't specify -- a certain employee, but more generally, conveys to employers what are best practices. >> I think unfortunately, we're -- okay one more question. Wittenburg. JODY: Go ahead Dave, really quickly, if it's a quick answer and quick question, you can do it. No you're out of time. All right. He'll talk to his colleagues later. Thank you for a great session and see you back here in 15 minutes for the last one. [APPLAUSE] (2:16:34 p.m.). Panel 4: Lessons from the private sector and the Veterans' Administration on increasing employment. DENISE: Okay, they're telling me we should go ahead and get started. (2:34:23 p.m.). DENISE: So we will do that. This is the last panel of the day. And, again, I'm Denise Rozell from AUCD. I am the moderator for this panel. And I think we'll just... I guess we'll introduce each person as -- now, everybody now? Or should we go forward? Doesn't matter. Okay. So immediately to my right the first speaker Tim Tansey from the university of Wisconsin, talking about a case study in inclusion of people with disabilities in the manufacturing setting. Reagan Baughman next to him from UNH. On employment incentives in SSDI, lessons from the VA disability program. So we have a veteran's piece to this one. And then John O'Neill and Elaine Katz who should be this point in the day be familiar to all of you from the Kessler Foundation, talking about the supervisor perspectives and the employment disability survey. So there is a time keeper in front. So I'm going to let you guys get started. Tim, it's to you! >>TIM: (Approaching) good afternoon, I'm glad I made it here, finally. [LAUGHTER] >>TIM: Fortunately, I got out of Madison, just before the 10 inches of snow fell last night. FROM THE FLOOR: Yay! >>TIM: Excited to be here, excited the kids have finished shoveling the-degree at this point. I've been told there's supposed to be another 6 inches by the time I get back and that's on me. Yeah, so today I want to talk about a bit of a -- one study under 1 phase of 1 project amongst many projects. My general presentation -- I'm going to make you do work and start off with softball questions. Why do employers hire people to work for them? What are employers looking for? Why do they need people? >> Productivity! >>TIM: What kind of productivity. >> Good productivity. [LAUGHTER] >>TIM: That's a variable. We want to give them good productivity. But they need some type of service, product. Something to be produced. Every employer out there, is essentially looking for the same thing, they are looking for people to do work for them to help them accomplish some greater goal. Now, we've all worked with different employers, but we describe maybe every employment instance we've been in comparable to the one before it? Have we had good employment settings versus maybe ones we kind of talk about at parties now saying let me tell you about a job I once had? >> Yeah! >>TIM: So what's the difference between those? What -- when you think about a good employment experience, versus your own experience as far as one that wasn't so great, what are some of the defining pieces of we'll keep it positive. What were the defining components of your positive employment experiences? Yeah. >> People, you know, it's the people that my colleagues and my coworkers collaborators people I work with. >>TIM: First of all, they're fun to be around you enjoy their company. They're part of a team. So they're promoting each other at the same time as we're working together. Of course, what else? People is one thing, but there's other things we look at as far as why we stay in a job versus move on. >> Actually able to do the job. >>TIM: You can do the job. >> Resources, allow me to have the scope of my employment. >>TIM: You had a sense of what you were doing, the capacity to do it and resources to make it happen. What else. >> Having an impact. >>TIM: What does that mean? Got to clarify. >> You believe what you were doing matters to something beyond you. >>TIM: All right. Great purpose. You can see the greater vision, of what you're trying to do. And it's not just you accomplish what you set out for a task for that day, but you found that it had some greater benefit, whatever it was to the broader communities of people around you. There was one person in the back there. >> Compensated fairly. >>TIM: I'm still looking for that to happen. [LAUGHTER] that you're compensated. You feel like the -- what you're -- what you're being paid is consistent with what you should be paid for that type of work. In the back there. >> Inclusivity. Someone with a disability, I always, like to look around and see are they actually hiring people, all different scopes of people? >>TIM: And I agree completely. We start thinking about employers, and I want to talk today a little bit for the -- slides are boring. I've been told when I present watching paint dry is an enjoyable experience. Bear with me on some of these things. But in thinking about the employment setting. One of the things we can look at under the workforce investment opportunities act, we move from can we get somebody into a job to can they retain that work? Is there a long-term benefit for this individual? Are they doing better over time? Are they increasing in wages, in hours? One of the important things we're looking at under the rehabilitation research training center, on employer practices -- is what is going on with those employers, that we can talk to and identify who is more likely to hire? And the reality is, that we're finding out fairly quickly, the employers more willing to hire people with disabilities are setting up workplace environments that are likely to retain those individuals over the long-term. A little foreshadowing of things I'm going to talk about in the remainder of -- I've got ten minutes. I'm good. So I want to talk a little bit -- try the button here. There we go. So one of the projects we're doing under is going out and talking to employers, not sending a survey, not asking them to fill out a piece of information here and certainly, I mean, when you look at the overall outcomes and how many people they've hired over the years but actually going into employers, and identifying what are you doing? How would you represent your efforts to hire individuals with disabilities? Why is it important to you? Why is it important to all the other individuals working at this -- at this job? At this employer? What is the experience of individuals with disabilities at that employer, if the employer says oh, we're very welcoming and we get kind of something different from the employee, is this really a conducive supportive, what we call -- is it a good temperature to this workplace? When we talk about long-term success? So we're in the process of doing a number of these case studies. 9 is kind of the minimum we're looking at actually doing additional ones of these. Where we're going out and talking to employers, not just one type of employer, looking at employers of different size, because what a small employer -- employer who has less than 50 employees might describe as being how we approach things doesn't work for the corporation with 10,000 employees. And vice versa. Also we've identified very quickly is what is useful in a manufacturing setting, may not work so well in a... other -- customer service setting. It may not work so well, in a technology setting. It may not work well, in a healthcare setting, based on what -- how those different areas of those employers are set up in those different industries. So we've gone through this process with one employer so far, we have a number of additional of these case studies scheduled at this point, but I want to talk about the first one. Because one of the things when you first started putting the project together is what are the areas we might want to know more and coincidentally started looking at what has been the rise in manufacturing in the United States in the last four years. Trying to identify particularly employers that are related in manufacturing, that may be an initial step for a number of individuals with disabilities. So we conducted a group of interviews with individuals from all different levels of the company heart-Schaffner Marx, a plant in Des Plaines -- does anyone know what they produce? >> Suits. >> I had the foresight, I had a meeting the CEO, I bought one of their jackets and when they asked me what I was wearing, I said, of course, them. We had a number of interviews with the CFO, CEO, the leadership of the organization to midlevel managers who are actually driving different decisions responsible for the areas, all the way down to the workers doing the work on a daily basis, the individual who is working in the sewing department. The individuals working on the loading docks. Who are -- managing their -- their Internet platforms? To find out how they have perceived individuals with disabilities as being a part of the workplace. So we use five interviewers, three different coders and an auditor to look at both collecting the data, identifying kind of the major pieces of themes that were coming out of the information from the qualitative standpoint. And then ultimately someone to evaluate here's the thematic representation of that representation, moving forward this is what I'm going to talk about. And -- if I don't hit it often enough. There we go, over a two-day period we interviewed 33 Hart Schaffner Marx in the autism workforce, it was a private company that was working with Hart Schaffner Marx to try to help them look at their inclusivity of persons with disabilities, as I mentioned upper level management. Being able to talk to what the CEO, the CFO director of HR telling you about how important the hiring of people with disabilities are and compare that to those supervisors, the person who is responsible for the loading dock, the person who is responsible for the sales department. How do they express it? And then look at that in relation to the supervisors, that ultimately coworkers and employees with autism and individuals without. There's some domains we identified and jumping ahead to the domains, I only have what ten minutes -- 12, 5? [LAUGHTER] >>TIM: I was told that I had 15 minutes just to -- just to drone on, but was interesting there were certain domains we saw across, whether it be the management team, the supervisors -- or the employees -- and certain ones that were specific to the management team. From a -- looking at the groups across all groups, shares across groups looking at how they evaluate autism workforce, approach to inclusivity of persons in the workplace. We identified the impact of the influence of the autism initiative on HSM and its employees and I'll describe these in greater detail. Performance integration of people with autism and the characteristics of Hart Schaffner Marx, what is it about this employer that under the influences its success of this type of integration program? And certainly talk about from the management perspective, is interesting hearing how they would like to see this program proceed in the future. And grow over time. So when you think about evaluations of autism workforce, secondary company working with them on inclusivity -- was the idea of anxiousness. The idea of being hesitant and for both employees, for supervisors, about the hiring of persons with disabilities. When this program first started. It was the idea that we're not really sure what's going to happen. We're being told from a -- a top-down model, that this is an area that we want to increase the company profile. We want to be looked at as an inclusive company. Now, when you actually talk to your midlevel manager, and you talk to the CFO, they all said we're not sure if this is going to work, if this is a good idea. Now, they changed their tune over time, but still, when you think about the original hesitation, the initial resistance that they experienced, in looking at adopting a fairly-inclusive program, we have identified, again, across the board this same hesitation. We've heard it time and time before as far as employers not wanting to hire somebody because they worry about productivity. Fitting with the other employees is this person a good social addition, we worry about whether or not there are going to be other issues as far as what their needs are going to be as far as workplace accommodations, all the things coming out in the initial interviews as far as these were the pieces we were worried about first and foremost. What was interesting to them, they said, but -- they would say we worry but. And I'm always looking for the "but" because it essentially negates everything they just said, we were worried but now we have a very different perspective. They talked extensively about the benefits of the autism initiative. And this is kind of our phrasing of it, because this is how they say we're really excited we're doing this now. Really excited we're engaged in this. And I think it was over here as far as that sense of greater impact. It helped the company but it helped me personally. It gave people a sense of purpose beyond -- I come in every day to help make a suit. I come in every day to work on the sales floor, or to walk, talk with different wholesalers about buying our product. Now I come in and the company doing all those things but I have identified we are a disability-inclusive workplace. I feel good about that as being a part of our mission. That we are not only looking at the individuals who are... are going to be selling our clothes to, both the people working with us as well as outside but ultimately that we're living that talk. We're actually being inclusive in our workplace as much as we are trying to be inclusive in who we sell to. I'm not going to read through all the quotes, because they're up on the screen and I know I've only got -- is that 5? It's kind of hard for me to read from here. [LAUGHTER] >>TIM: When you think about this, it was great to hear a member of a management team, who talked about this, as -- it makes a better work experience overall. I didn't do it! (PowerPoint goes blank). >> You joked about the time! [LAUGHTER] >>TIM: Is this what they do? I mean, it's better than the hook! [LAUGHTER] >>TIM: I told you watching paint dry is so much more fun! [LAUGHTER] >> Sorry! >>TIM: That is okay. I will keep us going, we're at zero minutes, but essentially -- here we go (PowerPoint back up) I only have three slides here, I'll be very, very quick. >> You're good. >>TIM: I can't do anything, the clicker won't work until I think -- view, show, no! [LAUGHTER] >>TIM: Tim Tansey hamming it up! [LAUGHTER] I do have to be respectful of time and my copresenters without going too far in it. You can go ahead and pull up the next one, what we're finding and what was clear -- before they started this type of inclusivity program, if you actually ask them what was the environment like? It was not an inclusive environment. They were not necessarily engaged, when you talk about providing accommodations for anybody, individuals with short-term impairments. Individuals who are having some limitations -- it was that was a burden. That was something we didn't necessarily want to look at. It affected the overall productivity of that department. Subsequent to this engagement, an inclusive workplace you actually had the sense of pride. You had this sense of warmth around how do we support each other? Not just the individuals that have come in through the specific project; how do we support the individual in the workplace that is now experiencing cancer and going through treatment for cancer? How do we support the individual that's had a cardiac event, and looking at how do we adjust their experience in the workplace to make it -- one that's acceptable and inclusive of them? So in talking to the employees all the way from the CEO down, there was this push for inclusion. The adoption of the internal benefit of feeling like I was included did not happen until well into the process. But it is that idea of how do we get there from A to B? Without talking too much about it, because I -- you guys will ask me some questions at the end. I'm sure. It was the focus of the CEO CFO saying we are going this route by requiring diversity training, by requiring outreach and people to spend time, but it was a requirement of integration of persons with disabilities at all levels of the -- of the employer. It wasn't an enclave the idea we're going to hire the token individual over in this one area; it was looking at hiring individuals, as part of our regular process. Developing interview protocols to make sure we're providing accommodations, at the forefront of those -- of that process. And ultimately, having those opportunities for individuals to check in with their supervisor, checking with HR on a regular basis; that they could express concerns that they had, needs they had. Talking with family members, if family members were engaged; but having that component the broad communication strategy, ultimately did change the overall temperature of the employer to a point, where now, they look at -- if -- when they're actually going through their interview process, they're trying to identify who are the individuals in the group that may not have other opportunities because people aren't giving them a fair chance? And rather looking at things that had nothing to do with the jobs, restructuring their interview process to focus solely on those functional requirements of the job. And then within the part how we provide accommodations within the context of the job interview. All right. I know I'm out of time. So I'm through. DENISE: Thank you! (2:52:18 p.m.). [APPLAUSE] Thank you. REAGAN: I'm happy to be here today to talk about some work, some ongoing work. That I've been doing with Andrew Houtenville. This paper is a little bit different than some other that you've seen today. It is in some sense a policy thought piece about specifically about policy design. With -- for SSDI, and I would say more than lessons from the VA program at this point we're drawing ideas from the VA disability program. So... Motivation here, the focus is on employment and labor force participation of people with disabilities. And so I think taking for granted that the statistics are well-known to people in this room. We have decades of declining labor force participation, alongside decades of growth in the SSDI program. And although the findings aren't universal in terms of magnitude, there are a number of studies that indicate that the program is probably lowering employment and labor force participation itself. So it's going to be the motivation for the question that we're asking. One of the main way that economists would describe what is probably -- at least part of the incentive problem that is preventing people from working, or, you know, participating in the labor force, we would do it in terms of we call budget constraints. Thinking graphically about the relationship between people's total income. Which is on the vertical axis. And the hours that they work on the horizontal axis. So these are annual numbers. And so this is just plotting out what that relationship would look like for someone who earns $15 an hour. In orange. Someone without SSDI, so they start at zero and accumulate $15 per hour. And although you can't make the graph show it easily that would go up all the way to the end of the graph, sort of underneath the blue line. The blue -- navy blue line, is the SSDI program. And you see this famous cliff effect, where someone can keep the full SSDI benefit. But only up until the point of reaching SGA. Okay? And so that -- in this set of numbers would occur for this person, $32,000 and change. So basically working halftime, but the incentive is pretty stark when you look at the fact the same person without SSDI would have to be working, or even if -- the person past SGA kept working they would have to work twice -- another 20 hours, essentially, to get to the same level of income. So there's a clear incentive problem. And this has been a focus of a lot of talks about policy design. This cliff point. This is what economist would call a "substitution effect incentive" and that's at the point of a person reaching SGA earnings level. All of a sudden, the monetary return to an extra hours of work, it doesn't -- some other programs doesn't just diminish or even go flat. There's a big negative shot all of a sudden you lose a lot of money. And people have realized that and so there have been projects like the benefit offset pilot project, which reduced benefits gradually, not going to show you another budget constraint, but basically it took that cliff, and smoothed it off to some extent. Taking away one dollar of SSDI benefit for every $2 earned. And there was hope that that might create some momentum, but evaluation has been done, but no effect of probability of having any labor market earnings. So we're thinking about an alternative model and that is to borrow from the structure of the veteran's administration, disability program; focusing -- it's pretty simple because it's focusing on two key components of that program that, we're going to argue, both make it -- could make a difference in labor force participation of people who would otherwise be in the SSDI program. The first is the benefit -- if this is not an all or nothing program. Not like SSDI whether you either get on to the program or don't. Benefits are given at different levels of generosity from zero to 100% depending upon the degree of impairment that the veteran's association examiners believe that you have, we're going to focus on one to 100, because people at zero percent get some services but they don't get the cash benefit. And the second... Is the lump sum -- monthly benefit isn't reduced by earnings or employment? So if you qualify, medically, you have your degree of impairment -- it doesn't matter what your wage is or how much you work; you have that benefit. So basically that last point is going to reduce, it's going to take away the cliff effect and I'll show you a diagram of what that looks like in a second. So the VA -- one of the big characteristics is they have their distribution of ratings. This is the overall distribution; so 10% low percent impairment benefits are very common. You'll see in a minute a lot of them are musculoskeletal problems. So 10% -- and when you look at the full distribution sort of dwarfs everything. This one's a little bit more helpful to me. In our analysis, we look at total numbers, everybody on SSDI may also look at -- on the VA -- we also look at the four most common sort of average across the two groups, types of impairments, we have mental in green. Neurological is the next one. Cardiovascular and then musculoskeletal and you'll see those 10% benefits for musculoskeletal, so cardiovascular problems. Neurological looks a little different. We don't quite understand why, but they tend to rate people at 100, 70%, 50% or 30% -- a lot of them are sort of jumpy. Not a lot of people get 90 or 80% ratings but you see what the distributions look like. So, the second part of the argument, why a VA design might spur more labor force participation -- is not just reducing the cliff. Because people keep their benefits. They don't have their benefits fall to nothing, at a certain amount of earnings. It's an income effect. What an economist would call an "income effect" and that's when people's incomes rise -- and this isn't from their earnings necessarily, but, you know, income in the household from a spouse for example. When that income rises, they're less likely to work; and when that income drops they're more likely to work. And so another effect of this benefit system, some people would -- would keep their 100% benefit levels, but some people would get benefits at a lower level. And so all else equal, we would expect that to also have an effect on labor force participation. So this is -- I probably should have put this first. This is, again, thinking about the budget constraints. Now you can see that cliff effect for the -- current traditional SSDI program. Against a very simple VA benefit budget constraint for somebody who -- 60% of the full VA -- I'm sorry, the full -- 60% of the full SSDI benefit. And so they basically, look like any other wage-earner; they just have some extra income. Okay? But significantly less, in -- could they have $8,000 compared to 14,000, some, dollars because of that discount factor? So one thing that we would, obviously, be curious about is how to estimate this income effect. The interpretation out this is something we're still working on the literature, it's not terribly helpful, but in 2001, the veteran's administration announced it would pay benefits for people who had diabetes because there had been clinical evidence that agent orange exposure was linked to diabetes, so there are a whole bunch of people who automatically responded gained benefits and they found 3 percentage point decrease in labor force participation. They don't actually provide a good base for that. But I think 6%, is a relatively decent conservative number for that. So that's going in one way, but it would also suggest for some people using, up to -- you know, going down to 60% of their benefits or 70%, would spur an increase in employment. So the last thing that we're going to talk about, or talk about before some concluding comments, and what we're doing right now -- we have done a basic simulation of what not the labor supply effects but what the budgetary effects and what distributions benefits might look like under this new system. It's important to keep in mind that the system change would produce two things -- would happen, one would be that some people who currently have SSDI, some would be 100%, but some would be less than 100%. But there would also be a number of new beneficiaries, who are currently not getting SSDI, maybe applying but lower levels of impairment -- they would be gaining a benefit. So because some of the new people would be getting such a small benefit, we -- we hypothesized an employment effect should still be positive. It's less clear what the net budgetary effect would be. So, again, this is a -- a simple version -- I'll tell you about a -- what we were going to do to try and make it more realistic. But first thing, is to take the number of SSDI recipients, assume the degree of disability required to get into SSDI currently, is approximately equivalent to 60% on the VA scale, and I think we have some citations to back that up. Then we take the distribution of the VA percentage ratings, between 60 and 100%, we see where people in the VA would fall, and we just apply that to the SSDI population, because we don't -- this is one thing we're lacking, is we would love to have better data about types of impairment, degree of impairment but we don't have that. So we're simulating using the VA. This is what that would look like. And you would see... again, -- you know, number of people would still -- 40% in a lot of categories just under, would still have 100%, but there would be a lot of people, for example, with neurological conditions, that would be at 70%. Based on the VA distribution. And then we just use, we use average SSDI benefit and we -- using the numbers we get in the discounted categories, we apply to percentage, and so for people who are current recipients, spending would fall on the aggregate by 2.2 billion dollars. The second effect, though, would be for people not currently receiving a benefit. So we impute a denied number for SSDI, assume that all denied applicants get a partial benefit; somewhere below between 50 and 10% and then, again, we apply -- excuse me, the VA distribution, to that number. And we see not surprisingly, a lot of people getting 10% benefits especially for musculoskeletal complaints, but also get -- for whatever reason, the way they apply, the neurology impairment percentages, they're jumpy, 70, 50, and 30. Again, use the total benefit and discount it. And this causes spending to rise by 1.46 billion dollars. So this is what our -- our simple simulation would suggest. We've done this with all beneficiaries. And then we have broken out mental neurological cardiovascular, musculoskeletal, because they're large categories. It's interesting to see that there's net savings of 762 million dollars. Overall. And in musculoskeletal: A lot of that because there's lower degree of impairment -- but for -- in some categories, mental health, for example, there might be more people with 50 and 30-degree percent impairments who might not get a benefit now, would be getting a benefit. So our conclusion, tentative conclusion here, is that there could be two advantages to using the VA, some part of the VA system design, in SSDI, partial benefits, and benefits that are not phased out or cut off, at certain levels of income or earnings or employment -- they're likely to generate stronger work incentives. Using a different margin of incentive, instead of that substitution effect. You know, what's the return to the extra hour? Manipulating the base income. And they're also likely to generate budget savings even after taking account -- new beneficiaries and that's interesting because that could be used budgetary sense for trust fund issues, it could also be used to provide other services that people have talked about today. Caveats are important. Bunch of -- assumptions that are basic simulations make, that the degree of impairment is -- equal across programs, we don't have the programmatic data to be able to tell that. Also, that the degree of impairment is not correlated with earnings history, which determines your actual SSDI payment. That -- this is an important, the number of applications to SSDI doesn't change under the new system but there's reason to believe that it would; that there might be this woodwork effect of people who have a disability who would apply. And there's also in the VA system, they have -- sort of base benefit but if you're at 100%, they give you an extra thousand dollars so we don't include there's no -- I think, a priori reason why Social Security would add onto the full benefit. But if you do, it becomes much more expensive. Right now what we're working on is getting a better understanding of what the degree impairment of the current SSDI beneficiary group or at least, group of people who report disabilities look like. So that we can understand what's going on with current beneficiaries, how impaired they might be. As well as the bigger unknown is what the new beneficiaries that -- might come into the system would look like. And so we've got pretty good data in the topical modules about not just... indicators for disabilities but it has a pretty nice list of -- what medical condition caused the disability. Which would match up nicely with this. And we can break it out, and look at age distribution earnings distribution and things that would help us to refine some of those simulations. Thank you! [APPLAUSE] >> That's interesting! (3:08:11 p.m.) >> John O'Neill: (Approaching) okay. Elaine Katz and I are going to talk today about the 2017 Kessler Foundation supervisors' perspective survey. The purpose was to identify practices used by employers, to increase the employment of people with disabilities, and the effectiveness of these practices. I'm going to move pretty quickly, because this is a data-heavy presentation and we want to talk about the implications at the end. One of the -- one of the things we wanted to address in this particular survey, was, the social desirability that was inherent in other employer surveys that have been done in the past, where the participants knew that those -- doing the survey knew the organizations that they were working for. And there's an inherent social desirability, I think, in getting responses that way. We focused on supervisors. Who did not -- who knew, we did not know where they were working. And we wanted to generate actionable information that could support the adoption of promising practices in the future. We had -- a little over 6500 supervisors from across the country, drawn from a Qualtrics, business to business panel. And we were focusing on recruitment and hiring on-boarding and training, retention and accommodation. We wanted to know the processes and practices as -- and their effectiveness. We wanted to know, the supervisors and others upper-level management's commitment; and we also asked a lot of open-ended questions. The demographics of the respondents, the majority were female. The majority were white. The -- we had good representation across the age groups. But most of the folks who responded, were between 35 and 54. And over 5,000 of the participants had a college education, or greater. Which is to be expected, given that they're supervisors. Company size, we had a good distribution. The largest little under 3,000, were large organizations with 500 or more employees. And -- but we had a good representation of small organizations, small employers, as well as midsize. We're first going to talk about upper-level management's commitment. And the importance of -- of hiring et cetera, to the supervisors. And I'm going to move through this fairly quickly. As you can see, upper -- from the perspective of the supervisor: Upper-level management's commitment to hiring people with disabilities, was about the same as the supervisors. In terms of supporting people with disabilities, learning the job, there's quite a difference in terms of how the supervisors felt about the importance of people learning the job versus their perspective of upper-level management's commitment. In terms of providing accommodations: Supervisors also found it much more important to provide accommodations than they felt upper level management's commitment was. We're going to talk about organizational processes, whether there was a process, whether it was effective; whether it was as effective for people with disabilities. A large proportion of the organizations had a process. A large proportion of those who had a process felt it was effective, the supervisors did. And fewer, about 61%, felt it was as effective for persons with disabilities. Hiring goals: 57% of the supervisors said that their organization had a diversity hiring goal. 28% said they had a disability. Hiring goal. And only 12%, included disability in the diversity initiative of the company. I'm going to move on to process, supporting employees, learn the job. 86% of the supervisors said they had a process a large proportion, 93%, said it was generally effective. And 73% said it was as effective for persons with disabilities. Requesting accommodations: 66% of the organizations had a process for requesting accommodations. And 96% of those that had a process, said they were effective. Centralized accommodation fund: This is interesting, only 16% of the organizations had a centralized accommodation fund. But 94% felt that of those, felt it was effective. Specific practices: Partnering with disability organizations, that is done frequently in communities that are attempting to -- or -- and organizations, employers, that are attempting to recruit people with disabilities. Only 27% of the supervisors said their organization, their employer had a -- partnered with disability organizations. But those who did, 95% felt it was effective. This is another question we ask for those who didn't have it or didn't know: We asked whether it was feasible. And a large proportions of the supervisors felt it would be feasible. In terms of training supervisors and accessible application and interviewing techniques: Tim was talking about that. Only 40% of the supervisors said their employer had such a training. But those who did, 87%, said it was effective. And, again, large proportions of those who didn't know, or said no, they didn't have this training opportunity, said it would be feasible to -- to implement. Hiring practices. Return -- review or audit hiring procedures. Only 47 -- 43% [sic] of the supervisors said their employer had this practice. 86% said it would be effective. And a large proportion also said didn't have it or said -- said it would be feasible to implement. Short-term, outside assistance. Such as job coaches, et cetera, only 19% of the organizations said they used this definitively, and 25% said sometimes. Those are used sometimes, or definitively used it large proportions felt it was effective, generally. And fairly large proportions felt it was as effective for people with disabilities. And 70% of those that didn't have the practice, said it would be feasible to implement. Job shadowing. Large proportions said, yes, they definitively use it or sometimes. Large proportions who did use job shadowing, said it was effective. And fairly large proportions of those said it was effective for people with disabilities. And a fairly large proportion -- of those who didn't have the practice said it would be feasible. Looking for time... [LAUGHTER] >> John O'Neill: In terms of training, having outside training -- I'm sorry having on-site training by a supervisor, or coworker: Large proportions of the organizations did this sometimes or definitively said, yes, they did this. Most felt it was effective, of those that did have the practice. And high proportions of the organizations felt it was as effective for people with disabilities. And 77% of folks who didn't have this as a practice, felt it would be feasible. Job sharing: I'm going to move through this fairly quickly. As you can see, there were large proportions who had job sharing as an accommodation practice, but they felt it was generally effective, and as effective for people with disabilities. But fewer felt it would be feasible. In terms of flexible work schedule: There were fairly large proportions that used it sometimes. Or frequently. And they felt it was generally effective, and effective for people with disabilities. And lower proportions felt it would be feasible. Working from home: Small proportions felt, or used it definitively; or some -- sometimes they felt it was generally effective. And they felt it was as effective for people with disabilities. But few felt of those who didn't have it felt it would be feasible. I'm going to move right to the key take-aways. Because, I think, that's probably what's most important. And that's Elaine, who is going to talk about that. >> Elaine: (Approaching). >> Okay. So, you know, why is this all important? Why are we doing all this work? And as we mentioned I'm not really the data geek on this team. I'm always the context person and how does it relate to the hiring of people with disabilities? And really, the survey is important because it's identifying key practices, that we can use in the field. And information that organizations can use. As they do their work. So as you're looking for the commitments and importance, it really reinforces, that nothing happens without effective directives from the top. And we even saw that today in Tim' s information that Hart Schaffner Marx didn't do anything without the CEO. So, you know, when you're talking about things like organizational management it's fine for a provider agency to talk to a Maria manager and maybe they'll hire one person, but if it's a substantial company and you want more than one person hired, it has to come from the top, even if the company is saying oh, yeah, we all share in management decisions -- that's not going to happen. And it's really important, it's not surprising, that supervisors, were really saw it important they were being judged. Attach much importance for learning the job and providing accommodations, why? Because they're ranked in their own performance is depending on the people who work under them. So they only share their team does a good job, they won't get a good rating. So it really -- we saw throughout -- throughout this whether it's accommodations. Or learning jobs or on-boarding whatever it was, the supervisors could be much more invested than anyone else, who's directly affecting this. And when we're talking about effective processes -- you know, companies do have some processes in place. And they believe those are effective for people with disabilities. If you talk to people in the field, they'll say they're not. You have algorithms. You have all kinds of obstacles for people with disabilities. But when companies say they have an accommodation process, maybe they need that provider in the field to work with them, to say well, you need the accommodation on the interview, forget the job, I mean, maybe you need an interpreter, somebody to help read an application. Maybe you have to have somebody who interfaces better and has more sensitivities who is the initial screener in the company. So I think you have to think differently and out of the box, to be able to use these in the field. When we're talking about specific practices, this is my favorite. Because the survey showed that only 27% of organizations, big companies, use providers for help or other assistance. But when they did 95% found it effective. So what does that mean for everybody who is doing job placement and working in the field and their providers? It means you need to convince employers, by using this kind of data that if we come and we help you, it is going to help diversify your employment practices, you -- we are going to make a difference. And I know they may not want to be on-site, there's been a lot of talk with employers not wanting job coaches there a lot of times, but at least initially to help them meet the candidates and work through the recruitment process; that can really help. You know, Amazon; Sears worked with Indiana, Cintas -- our project involved Pepsi and Ability Beyond -- so companies are looking at getting that help to what they need. And also, skipped a slide but that's okay, when we're talking about common practices with universal design, I think it's really, really important to say, yes, there are lots of things we can do, we need to help providers think differently and think more objectively of what their value is to the employer, and to talk their language. We've heard a lot about it. But it's also the CEOs and advocates in the field, randy Lewis was one of the first really to not only start the initiative at Walgreens, and we've taken him on the road to talk to other CEOs but he has a summer home in Illinois and every summer he invites 5 or six people that are key in the corporate field can to come and talk about employment. UPS, best buy, Office Depot. What does it mean for future studies looking at the survey, obviously, we've collected lots and lots of data? And we got lots of comments, and that's always the most difficult to put together, but we're really interested in doing a deeper dive, into all the comments, so we can learn, you know, u what were some of the really hidden gems that the survey provided. And we also made an option to contact the respondents at a later date and hopefully we may be able to do that. So thank you, all, for -- letting us come before and talk about the survey, and if you want information, please e-mail John and I. You can see our contact information. [APPLAUSE] >> Thank you, all, now we have -- I show maybe seven minutes for questions. There are mics in the room. If you would raise your hand. Or if there's anybody online with a question, I'm sure they will tell me. [LAUGHTER] FROM THE FLOOR: Q. Here we go. Leslie Kaplan from NIDILRR. Tim, this is a question for you? A. Uh-oh! A project officer so I've got to be careful. Q. I've never done this before. With a grantee, but it's -- don't worry. So? [LAUGHTER] Q. This morning, we spend some time talking -- there were I think a couple of presentations that talked about employment quality. And I think what's going on in -- for a couple of members of the panel, this panel, we're talking about employer quality. And you mentioned a few times in your presentation, the idea of employer temperature, and I wonder if you could let the -- give the audience some idea of what you mean by that? A. Tim: Of course, when you think about employer temperature as an audience I think you actually described it pretty well based on the questions I started off with. One of those except for purview, was -- we're going to look at money, which is important. But when we look at an employer, it isn't just the CEO giving a direction saying we're going to be a disability inclusive workplace. It's to what extent have they set up the policies the practices to what extent do the other employees look at disability, as being an equal member of our employment community? Versus being a secondary group? Versus being a -- an enclave that we're going to accept because it looks good on -- somewhere on paper. So we think about that disability that employer temperature component that is part of -- we're looking at under the RRTC, what is the environmental piece, when we go to a job, what else is the job giving us as far as are we happy to be there? Are we happy to be around the people? Do we feel like we're accepted by those individuals? And how does that acceptance come through? And, again, in the case of measuring temperature, it starts with the things we look at as far as helping people be retained long-term, we think about those -- those flexible work strategies, the flexible accommodations. To what extent do those show up in the recruitment. To what extent do those show up in the interview processes and it's really the pervasion of those policies throughout the company versus being isolated components. One of the other kind of parts to that is to what extent, not just that great information is interesting, doing interviews and a couple of other focus groups for other parts of the project, where you end up talking with folks and they'll say well, I think we have it, but if you ask the typical employee they wouldn't know what you were talking about. So to what extent do we make all employees at interview, at hire, as part of an ongoing piece to remind them, here are the policies that we have in place to support individuals from a diversity standpoint, but for individuals with disabilities. As part of that diversity. And it does show up in thinking about that climate, that temperature of the company as far as when someone goes to work at a warm place, is it a safe place, versus -- I'll say opposite that cold isolated piece, where they're a number coming through and the idea is to get them in and out the door and get them to get that productivity kind of piece done. >> I have another one back here. FROM THE FLOOR: Q. Daniel Davis from ACL. Given that -- given that disability is mentioned in terms of employment for, you know, -- for reporting for federal contractors; and that federal hiring initiatives have highlighted disability, to what do you ascribe the lack of acceptance in the business community that disability is a portion of employment diversity? That -- because that number jumped out at me, when -- when I heard that. DENISE: I think John or Elaine that's for you. A. John: I can try. Well, I think -- I'm not going to answer it directly at first. I think when disability is made part of a diversity agenda, within a corporation, that has the capacity to harness, all the -- all the aspects of -- that a company has built on in order to create that diversity agenda. And we happen to be engaged with a fairly large corporation. A pharma corporation. Whose -- whose HR director, as well as the compliance officer, within the corporation, has -- they have embraced the diversity agenda, and the -- the inclusion of people with disabilities -- people with disabilities are being included in the diversity agenda; they have a ways to go. They're working on it, but I think -- you know, two things: Upper level management is driving it. And, also, people with disabilities are now firmly ensconced in their diversity agenda. The other side of the question -- I don't think, I can answer. Maybe Elaine can. >> Elaine: I can add one thing when you look at the workforce development field in general that doesn't include disability, they don't include disability. So they look at diversity as your typical female LGBTQ, all the other categories. So it's a huge problem, and the foundation world is starting to look at ford foundation is starting to look at it. I think workforce development in general is starting to look at it. So I always say workforce development and disability employment, the agencies that that work in that segment are in parallel tracks they don't intersect. So until really, the why definition of diversity includes a disability. It's still going to be an issue. DENISE: It is 3:30, I am guessing we should end. So let me thank my panelists. And thank all of you, for your attention. [APPLAUSE] (3:30:53 p.m.). >>TIM: Thank you! >> David: (Approaching) Hi, everybody. I realize I'm the one standing between you and whatever happy hour, exists, or drinks, commute home. And so, I will -- I promise not to use the full half an hour but I do promise to give the summary of the day, and -- one of the first things that I wanted to do, was give a kudos to all the presenters today. [APPLAUSE] >> (Continuing) especially, you guys clapped too early. Not even done thanking people! I want to give -- to give Deb and Andrew, a lot of credit for pulling together the agenda, because I think the agenda, not only was strong. [APPLAUSE] >> Self-clap, Andrew, was very good. [LAUGHTER] >> But it also helps to tell a story. And that's kind of what I want to close on today, as a theme here, in the last five to ten minutes. And as we reflect back on the panel, we had five panels today. We opened with a health and employment panel. We, then, had an employment trends panel. Which set what I consider to be the context for the employment policy research. As the third panel, right before lunch we heard from nTIDE, which is really about dissemination. And the fourth and fifth panels that we just heard about this afternoon, were really about interventions and lessons from intervention development, of sort of where we could go. So it's really the context, the dissemination, and where can we go, going forward? And so what is -- which should be some of the punch lines we should take from today and hear some of the at least the world according to Dave of what I took away from today? First of all, one of the connective themes is that there's a cautiously optimistic set of outcomes in data. And during my research career, I -- I haven't gotten to be very optimistic often; and so I like the idea of actually leading it -- with a punch line, and one of the studies that was the source of optimism when you look at Andrew's work and looking at the employment trends, there's been real progress, now, mind you, the progress is from the depths of the employment rights, but the rates have increased over the past few years from 28 to 29%, which is fairly large number. And so, there has been some benefits. From the recent expansion. As you think about the health insurance papers from the opening session, I know there's a lot of focus on well, which one is right? How should we think about this? But taking a step back, one of the very first things that should really strike you, immediately, is when you look at the ACA and you look at the uninsurance rates, the uninsurance rates drop from 21% to 11%, according to Jody. That is a very large number for people with disabilities. And there is evidence from other demonstrations that have randomly-tested the provision of health insurance. Health insurance experiment. And the accelerated benefits demonstration. And these experiments show, here's what happens when you provide health insurance to an uninsured population. Some of the things we may want to look at are similar to what we looked at there, which is there are health outcomes, they improve quite a bit. And interestingly from both randomized study -- this is very mixed evidence on employment, unless you combine it with other supports. So moving along to.... some of the other themes: As we move towards a new time theme and we look to the... the nTIDE series. In my 20 years, the access to disability data has increased substantially. And that's not only due to the compendium, which you're going to see tomorrow, but it's actually due to these efforts through the webinars. And it's through the lunch series, how do we get better? And Elaine and John talked about, how we're discussing data and how we're disseminating data and there are ways I'm sure that we're doing well, and there are probably ways that we can do better. And hopefully, we learned both about that today. Another positive punch line is on the interventions side; and this was the discussion today, this afternoon. So when you look at sally Rogers's presentation you really got to see some very positive and repeated effects, particularly from the IPS model for people with psychiatric impairments. When you look at Yonatan's presentation about early intervention you see the early potential for early intervention amongst different populations and providing the right supports. And that plays into some of the much more bigger picture ideas from Reagan and Dave about how to refine and reform the system, at a much broader level. And using ideas from the VA program, as well as, the -- the EES, that Dave talked about. And then in the afternoon session: Tim talked about how to think about interventions from positive side about inclusivity and how those simple concepts of making sure that people with disabilities are included in all aspects of the management decisions, are important in developing an environment that promotes employment. I'm going to come back to that point in a moment. And finally: Elaine just mentioned, in her talks about how CEOs like Lewis, are going to other companies and creating the networking, and -- in order to set up environments to improve the employment outcomes of people with disabilities. So this is all positive stuff, and it's an economist -- here with closing remarks saying positive stuff so this is unusual for me. [LAUGHTER] >> Dave: Of course, there's problems. [LAUGHTER] >> And what are some of the themes of the problems we heard today? And perhaps the No. 1 theme, that we heard sort of throughout -- embedded throughout some of these discussions was the concern about job quality. Megan, gave a presentation that highlighted the issues around job quality in terms of wage, in terms of satisfaction, in terms of other measures -- of safety of work, in terms of measuring of what some of the disparities are there. Even in Sally optimistic presentation, about supported employment -- you know, one of the punch lines is that the -- the impacts while significant, statistically significant, have not been substantive to move -- enough to move people from SSDI, to SSI, to long-term employment, without program dependency. And then Frances and Nathan: Also talked about what I will call, the human picture of -- of people with disabilities. Where they talked about, in particular, David who was the scuba diver, who wanted to work more but felt the constraint the practical constraints of the issues of -- of SSDI. And this is also another point -- I'm going to come back on some, but both the qualitative findings from Matt's study as well as Tim. And then, there are two final problems, that I've seen. That were echoed particularly in Dave Stapleton's presentation about the Herculean task, of making change. And there's another sort of subtle issue, that Leslie -- we were sort of chatting about, which is, the own self perception of whether change is needed. And as you look at John O'Neill's presentations -- one of the things that you see is 96% of employers said they have effective practices. Well, if you think your process is effective, that can be an -- a hindrance to developing interventions and experimentation. And that's a very, very important point. And so the question is, where do we go next? Because it can kind of feel, like, we're between a rock and a hard place in terms of developing new intervention supports, and figuring out what are the interventions that sort of matter the most and how do we get people to do those interventions? And here, I'm going to go with another quote here because as you know, I like quotes. But -- some of you may have heard this in some of my presentations but this is Kaiser Fund, who is a statistician, wrote a book, called numbers rule your world. And what I like about this quote, is it juxtaposes the title, about how numbers influence so many of the things we -- that we do in our daily life; with how to disseminate evidence. And what is written, successful applied scientists develop a feel for decision-making processes. Crucially, they repackage their logic laced message to impress their ideas on those who are more comfortable with intuition or emotion than evidence. So this is really, really important, because a lot of times we think all we have to do is bring evidence to the debate. But if people are not comfortable with the evidence, or they're not ready to receive the evidence, or simply do not understand it -- the evidence won't be effective. And this was where I want to come back to sort of the qualitative messages, particularly from Nathan and Frances. And when they were talking about their stories, today, of people with disabilities, of David, angelica, Jackie and Jessie, the messages that we can take from them, is a lot of the innovations that you heard from these qualitative stories, were incredible ideas of the person's own, amazing both self experimentation, but also innovation. You think about the person that I think it was Jackie -- that was the person that was figuring out how to do the RX testing. And holding down very well-paying job, which is amazing, going from job to job. And that innovation really matters. And so that's where -- I'm going to conclude with, which is not sort of a summary econometric principles or things we should do; but rather to focus on the messaging and knowledge management as we go in these meetings and think about what is next in employment policy. And at least sort of my ideal world, the ideal is to be taking a look at all of our interventions. And all of our programs and policies. And testing them. With an open mind. And I think there are three ingredients for that. The first ingredient, which is clarity. Making sure we have a clear stance of where we want to go. Employment matters, making sure that is carried through not just to policy-makers, but to the people that are providing the services. To the people that are getting paid for the services. And to ultimately, the people that are getting those services. Would people say, that the purpose of, say, the rehabilitation support they're getting -- is to improve employment? And so clarity is one ingredient. The second ingredient: Is just simply humility and here is something I give, Tim credit for. And, actually, Tim I'll give you credit in the next one, sorry, this is not your -- the humility is knowing that you might fail. This is my -- my take of why we need to test, because we don't know what will work. Necessarily. And going in not knowing the answers, but actually coming out with asking the right questions. And this doesn't matter whether you're a researcher, a manager, or if you're simply a parent of trying to figure out what works. And finally, engagement. And this is the point that I do want to end with, because I was very struck by Tim's presentations. And the reason why I was struck by Tim's presentation is he engaged the audience and he asked the questions of the audience. And it -- it -- for at least me was better than a cup of coffee, because I started to lean in, but it sort of is a subtle lesson of asking the right questions and lead people to figuring out so they can adopt those ideas. And at least for me, Tim, one of the things I found immediately, from my own personal perspective was I started leaning in because you were can convincing me your ideas were my ideas, which is kind of -- one of the ways we want to make change. And so on that important note, I would kind of like to end this Conference. And enter -- I turn it over to you to talk about tomorrow, Andrew? >> Sure. >> Dave: Or do I just wrap up? >> Andrew: I don't know. >> Dave: We're experimenting! [APPLAUSE] (3:44:28 p.m.) >> Let's give Dave a hand. Thank you David, at least one person was paying attention the whole day. [LAUGHTER] >> Andrew: It's always good to hear Dave's comments he's done this a few years in a row and it's always good to hear. Tomorrow we start with our compendium. Our new annual report as well. They're available outside if you're -- if you're not coming back. If you're coming back, we'll see you again tomorrow. I think registration at 8:30. >> 8:30. >> Andrew: That's what I'm hearing. And so thank you, all, for coming, where is Dr. Brucker, are you going to say good-bye to everybody. She says good-bye, Deb Brucker says good-bye to everybody online and everybody in the room, and thanks, everybody, for coming. [APPLAUSE] (Proceedings concluded at 3:45:17 p.m.) 1 1