EPITalk: Behind the Paper
This stimulating podcast series from the Annals of Epidemiology takes you behind the scenes of groundbreaking articles recently published in the journal. Join Editor-in-Chief, Patrick Sullivan, and journal authors for thought-provoking conversations on the latest findings and developments in epidemiologic and methodologic research.
EPITalk: Behind the Paper
Proposing the Observational-Implementation Hybrid Approach
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Dr. Justin Knox describes a 'practical' approach to integrate observation and implementation science studies to increase the public health impact of observational research. “Proposing the observational-implementation hybrid approach: designing observational research for rapid translation” can be found in Annals of Epidemiology’s Special Issue on Implementation Science in Epidemiology.
Read the full article here:
https://www.sciencedirect.com/science/article/pii/S1047279723000571
Call for papers on Implementation Science in Epidemiology:
https://www.sciencedirect.com/journal/annals-of-epidemiology/about/call-for-papers
Episode Credits:
- Executive Producer: Sabrina Debas (Episodes 1-18) and Sofina Tran (19-)
- Technical Producer: Paula Burrows
- Annals of Epidemiology is published by Elsevier
Hi , you're listening to EPITalk: Behind the Paper , a monthly podcast from the Annals of Epidemiology . I'm Patrick Sullivan , Editor-in-Chief of the journal , and in this series we take you behind the scenes of some of the latest publications featured in our journal . Today , we're talking with Dr . Justin Knox about his article "Proposing the observational- implementation hybrid approach: designing observational research for rapid translation . You can find the full article in the journal's Special Issue on Implementation Science at www . annalsofepidemiology . org . Dr . Justin Knox is an Assistant Professor of Clinical Implementation Science and Intervention at Columbia University . His research centers on HIV and substance use , with a focus on vulnerable and marginalized populations , including racial , ethnic and sexual gender minorities in both domestic and global settings . Dr . Knox , thank you for being here today .
Justin KnoxYeah , thanks so much for having me . I'm excited to be here .
Patrick SullivanSo I'm excited about your paper because it really brings together this idea of hybrid approach implementation studies , which has been an area that's of a lot of interest in implementation science world , with the observational study design which is sort of a staple of epidemiology . So you're really bringing together these two worlds . It seems like a lot of the implementation scientists are maybe epidemiologists , but I'm not sure that a lot of epidemiologists are so familiar with implementation science . So I wonder if you could start out just by telling us a little bit about what implementation science is about , and then we can talk about the hybrid .
Justin KnoxYeah, absolutely . So that journey , I think , was my motivation for writing the paper . It's like I'm trained as an epidemiologist and how come I haven't heard about this implementation science stuff , which ? Implementation science is the study of methods and strategies that facilitate the uptake of evidence-based strategies to improve public health or clinical practice and incorporate them into everyday practice . So it's all about getting the things out there that work , getting people to use them and offer them and deliver them to patients and people populations . So , yeah , I was thinking , why isn't it an epidemiologist ? Had I not heard of this field ? And one thing I came across was these hybrid approaches in the context of conducting clinical trials , but thinking that , as you noted , many epidemiologists mostly conduct observational research . So if we had our own hybrid approach in that context , could that make people more aware of the potential of incorporating implementation science into their work ?
Patrick SullivanSo , as I understand this , I think there's a piece of beginning with the end in mind , meaning that at the same time you're collecting data in an observational study , you could also be trying to pick up some preliminary information about if this association pans out and if we take the next step of deciding that it's causal , then could we be collecting some preliminary information about how you would bring a public health strategy to the world , even as we're asking the question about this association . Is that right ?
Justin KnoxExactly , yeah , I think , as we're trying to find out how prevalent are these issues ? How , what are the incidence rates ? Are there causal associations ? Could we make our research more actionable by , at that same time , starting to collect information about what we could do to address these issues , which in many cases there already exists evidence-based interventions and treatments .
Patrick SullivanYeah , that's a great point . So we're like you have maybe a health condition and you're thinking about evaluating some associations and an observational kind of design . But you may know at the time you're designing that observational study that there are treatments or interventions available . So can you just give us a couple examples of what kind of information you might collect during the observational study and how that would relate to an implementation strategy or how we improve health part after it's done and again , it's conditional on whether the association is true . Can you just give us an example of what some of those nuggets of information might be in the context of an observational study ?
Justin KnoxYeah , absolutely so . In my bio I mentioned I do a lot of work on HIV and substance use . We're conducting a study to look at whether or not alcohol use is associated with HIV related outcomes . So we know there are a number of evidence-based interventions that can reduce people's drinking behaviors and some implementation data that we could collect around . That would be what would be participants' preferences regarding how they would want those interventions delivered , what they want them incorporated into their primary care settings or would they prefer to go to specialists for those services ? Do they want them offered by peers or by trained professionals ? So preference data is one . We also know that transportability is a major issue , right ? Certain things work better in certain populations based on their underlying characteristics , right ? It might be more effective among older people or people who are willing to engage in treatment , you know , who have identified that they want to reduce their drinking . So those are all things that we could collect information about , while we're also studying how prevalent these issues are and whether or not they're costly associated with HIV related outcomes .
Patrick SullivanSo it seems like this is really just sort of an opportunity to think at the beginning of an observational trial and maybe , with no additional cost , get some more leverage out of the results . But , like in this situation , if I were doing the study that you described , I know some of the observational methods and enrolling participants and maintaining them in a cohort , but I'm not an expert at all about interventions for alcohol use . So what , like how would that play out ? Pre-study , or what would you suggest people do if they have these ideas ?
Justin KnoxYeah , absolutely . That's a great point and something I thought about too as I got into this field of implementation sciences . Yeah , we don't really necessarily know what the interventions , treatments are . I mean , a lot of times as you work in an area or field , you come to be more familiar with those , but in certain cases people might not be , and maybe we should think about expanding our study teams to be able to include that type of expertise .
Justin KnoxYou know there have been separate calls for increased levels of funding for these studies . You know , as we're expected to do more and more , you know we need more resources in order to bring these comprehensive teams together because ultimately , I think it can make things more efficient . As you noted . You know we collect information sooner in the pipeline as to you know , in terms of when it could ultimately be used to inform what we can do to address the situation . If you're investing all those resources to conduct a big observational study and you have those skills on your team to recruit and retain participants , you know perhaps we should expand those to also be thinking about what are we going to do with this information and including expertise in intervention and implementation science ?
Patrick SullivanI think this would be good . We'll have to like send a special invitation to this version of the podcast to our NIH friends and colleagues just to say , like seems like it's an opportunity maybe for supplements to people who may be asking these kinds of like associative questions . To say , like there's a marginal , like you could apply for a supplement to get some marginal funds to pay for some time a colleague and I think one of the strategies is trying to find somebody who has expertise in the intervention side and just say , like we're going to be talking to a thousand adolescents living with HIV or at risk for HIV , or like whatever the underlying cohort is . What information would help you do a better job of interventions if this association turns out to be true . So it may be relatively low cost but it might save NIH funding .
Patrick SullivanOne study to describe the association . And then maybe that colleague that we don't know yet comes , reads your paper and says like oh well , we should see if this kind of like what the preferences are on this . The other thing that like to your point earlier is that some of these things are not necessarily you don't have to be an implementation scientist to add some questions about preferences for format or potential , like theoretical acceptability of different kinds of interventions . So they may fit quite well into the kinds of surveys that we're already doing or the kind of question formats that we're comfortable using .
Justin KnoxYeah , definitely . I think this could be an opportunity to build bridges between people doing observational research that aren't familiar with implementation science and those more integrated into those spaces . But also , like you mentioned , it's not doesn't necessarily have to be overly complicated you could start to think ahead about . I think it's something that would align with what epidemiologists already want to do , which is conduct actionable research that translates to public health impact . So totally agree . And then the supplement model is actually how we've had our hybrid work funded thus far getting supplements to incorporate discrete choice experiments , preference elicitation methods . But ideally you could propose a study that is an observational implementation hybrid approach and get all those resources together in one application . I think it would be a little more seamless .
Patrick SullivanYeah , and having the observational implementation hybrid approach named and published and reference of all , I think , makes it more likely that you could pitch this to a funder and say , like this is a real thing , because it's already a real thing .
Justin KnoxYeah , that was totally . Our motivation too is , I think people are doing this work and we tried to acknowledge that in the paper . But sometimes having some discourse around this , having a discussion about its merits or its limitations , and having a shared understanding of what something means , I think can hopefully help it be disseminated and utilized . So that's definitely our motivation and I think we're hoping to follow up on this paper with some further guidance on how you might actually carry out , in a hybrid observational implementation study , some examples of where we think it could be particularly useful , particularly when conducting research among populations experiencing longstanding health inequities , where there might be more urgency than usual to really conduct research in a way that it's actionable .
Patrick SullivanYeah , I'm glad you raised this piece about health equity because this is one of the things I think we need to be asking ourselves about .
Patrick SullivanEverything we do is like would this kind of approach advance health equity ?
Patrick SullivanAnd so how do we optimize that ?
Patrick SullivanSo I see two things . One is that it may be that you would identify different , for example , preferences for eventual provision of a service in different groups , by race , ethnicity or by age groups , for example , that may be disproportionately impacted , and so then you could prioritize and say there's actually a stronger preference for cell phone based methods in the groups that are actually inequitably impacted by this health condition , and so we're gonna choose that as a first approach and if it's a little less preferred by the people who are less impacted from an equity point of view , like we can take that . So that's one thing is we can actually look within subgroups to choose the options that will have the best shot at reducing inequities . But the other is this idea of shortening the timeline , because health equity and health justice , they say justice delayed is justice denied . I mean there is a particular urgency in groups that have gaps in health equity to shorten this timeline and not have the books lay sort of end to end , but have them overlap a little bit so that we build that stack faster .
Justin KnoxYeah , especially , you know , in these areas , communities that have , you know , had less investment . It's important , I think , that we're as efficient as possible , as we've acknowledged this , and start to invest more . And certainly , I think prioritizing those contexts and doing , you know , conducting research in a way that it can immediately inform what to do , is a way to hopefully , as you said , move towards justice .
Behind the Paper
Patrick SullivanAll right , now we're going to turn to the part of the podcast we call Behind the Paper , and we'd like to have on the podcast sort of a balance between understanding your science but also talking about your process .
Patrick SullivanAnd one of the things that I want to start with for this manuscript is that you really sort of developed and socialized this idea and then you brought in a number of other people who work in implementation science to participate as authors . So talk a little bit about just like what the value is . You're an implementation scientist yourself , but you developed this idea and then you brought in some other folks , and so how do they contribute differently ? Maybe there may be some authors that you sit and brainstorm the concept with , but there may be another group who meet the authorship criteria , meaning like they actually contribute to the development of the ideas . They read and approve the manuscript like they're real authors , but they come in at a different stage and add something different to the final product . So can you say a little bit about how you use that those colleagues strategically in writing this paper ?
Justin KnoxYeah , definitely , it was a great paper in that regard . I guess I'll just provide a little context . This is the first purely sort of theoretical paper that I've written . I'm used to writing empirical papers that ultimately can end up being rather formulaic . You have a gap , you set up a study to address that gap . You analyze data , you interpret the results , whereas in this case it was sort of .
Justin KnoxI started learning more about implementation science , coming at it from the epidemiology side , talking with mentors . What is implementation science ? How could it improve ? Why is it important to our work ? How can we make contributions there ?
Justin KnoxAnd in thinking through that and coming with this idea , I had a great senior author , Calvin Gang , who really helped me formulate this idea and talk through it , and I was able to start writing things up and getting his feedback .
Justin KnoxAnd then , yeah , it's such a great opportunity when you come up with an idea and take that initiative to draft something . Then you can gauge other mentors and say how does this sound to you ? You have your own take on this field and how this could align and how we could use this approach . What might its limitations be , where gaps that you feel need to be addressed and discussed , and I really felt so supported and positive in the way the paper came together and the way that you read drafts and helped me edit it down and respond to reviewer comments . So it's really a great learning experience and a great mentoring experience for me as an early stage investigator , to bring all of my mentors and colleagues and collaborators together in an idea that I hope people think is a good idea and they decide to start using it , and it's a way to make research more impactful and useful .
Patrick SullivanYeah , I want to pick up on this idea of mentorship because it's present at every stage of your career . It's present when you're earlier in your career because you need it , and it's present when you're in your mid-career because you start having doctoral students who are looking for mentorship . So mentorship is an issue that cuts across all of our careers and so I wonder for you and I'll just say , as you mentioned , we know each other and I've worked together outside of the context of this paper , so I think you're someone who's been pretty intentional about reaching out and asking for mentorship from folks at different points in your career . So can you just say , maybe a little bit of advice for people who are here in every way a colleague I wouldn't call you an earlier career , I just call you a colleague . But putting yourself back in that time , what advice would you have for earlier career colleagues about seeking mentorship , like , how do you start that conversation ? How do you ask ? How do you know when it's over from your perspective ?
Justin KnoxYeah , yeah , totally . I mean it's a great question . It's something I feel . I've been so fortunate to have great mentors throughout my training and you get sort of used to it in a way that you're learning and I think a lot of times you're working with mentors who've received great mentorship themselves and have always been incredibly generous and willing to pay it forward . You know and you realize how you work together and there can be growing pains at times .
Justin KnoxBut I mean for me , as you , as you noted , it's always , you know , being willing and wanting to take on that initiative of saying like I want to sound this idea off you , I want to get your input on this . How can I cut this down ? How can I say this more efficiently ? What do you think about this ? How can we refine it ? So yeah , and I think the way academia is set up is we're constantly being put into situations where we can be mentored , whether you're on a training program or you're on an independent , you know , mentor , career development award .
Justin KnoxBut you know , I feel like you're always being forced to think of yourself in the context of a team , whether it's a team of mentors or a team of collaborators , to get a project done . So , you know , as I started thinking about the idea this for this paper I had my K mentorship team in place . I had people who were epidemiologists and who were engaging in implementation science who I would naturally be , you know , attending their lab meetings , talking with them about my ideas , working on papers together . So , you know , it came quite naturally to me , but I imagine , if you weren't in that context , maybe you didn't . You know , you're an institution where there's less resources .
Justin KnoxIn that sense , my experience has always been you know , don't be afraid to . You know , reach out to anyone . Like you're at Emory , I'm at Columbia , you know , there's people on that paper from all over the country , right , and I feel like , in a way , it's really like there are no borders in academia . You know , people are always happy to talk about good ideas and collaborate across Settings and projects and topics . Even so , you just have to engage them and I find you'll be rewarded .
Patrick SullivanSo I'm going to just take a small moment of personal privilege , because you talked about , like , how we've all benefited from mentorship and just in the last month or so my major professor from my PhD , Dr . Ted McDonald , passed away and he's just a great loss to those of us who cared about him .
Patrick SullivanHe passed away after a very long career and even 20 years after I graduated he would email me and say I read with interest your paper on and I did a bench science PhD . I went on to do HIV prevention . You know research and epidemiology , so it's not at all his field , but he still read the work of his mentees and reached out to us and he always had a like , a good question , like , even though it wasn't his area . And just recently he passed away and I had a chance to visit the folks in my lab who had supported me during my training . So I'll just say like I think these mentorship relationships , when they're done right , affect us our whole careers and I remember Ted was such gratitude and along those lines I wonder you know you have been very intentional and you sort of talk about this how do you identify someone you think you might want to be a mentor and how do you approach them ?
Patrick SullivanI just think it's really practical for earlier career people to know , like is it weird to just reach out to somebody ? Or , you know , do you look for a common connection before you do it ? Like , how have you assembled your mentor team , how do you find the people you want to mentor you and how do you engage with them ?
Justin KnoxYeah , I mean , I think everyone has their own style right .
Justin KnoxSome people are completely comfortable reaching out , some people like to have collaborations going on all over the place and be a little bit involved in a lot , whereas some people really like to dig in and work very closely with one person or one team , you know , and people can be productive across settings in a different ways , and in no way do I want you know my experience to be like .
Justin KnoxThis is the way you do it , but for myself , of course , I can speak to that and I feel like the most successful way has been to work those networks in a way , you know , like , ultimately we're not a small community of people doing , you know , pretty intensive research in certain areas and in this case , like , people bring expertise from different areas , but a lot of people know each other , they've worked together , they can make a reference for you , they can make a connection , you can meet up when you're at a conference or you can , you know , jump on a zoom and shout about an idea that you have or a paper proposed to work with their data , and I think for me that's been the most successful . I mean , there are some people that have , you know , just reached out to cold . I really enjoy your research . I'd love to work with you on a project , but for the most part it's really been . You know , I think Teo Samford introduced us .
Patrick SullivanYeah , and when you reach out to people call , do they respond ?
Justin KnoxNot all the time .
Patrick SullivanYeah , I'm sure .
Justin KnoxLike I only remember the people .
Justin KnoxYeah , I'd be like oh yeah , I never wrote me back and it's fine , right , like , you move on , they move on and maybe you reconnect , you know . So , yeah , you know everyone's super busy , everyone's doing , you know , things that I think excite them and motivate them and sometimes there'll be a connection and other times there won't . But I think if you just keep pushing forward with your ideas and your agenda and what you think is important and what you're passionate about , you know , I do think it ultimately works out .
Patrick SullivanYeah , I want to close just by asking you to think about . Maybe you know , like people who are their PhDs in an epidemiology program , you know people who are earlier in their academic careers or CDC or governmental careers or whatever , who are epidemiologists . But and for implementation , science is like a little bit of a black box , seems complicated . What are just a couple steps ? You might suggest that , like if someone says I want to , I just want to understand more about this . To think about , do I want to like develop expertise ? Do I need to know just to be able to talk about it ? So one thing they should read your paper of course , but beyond that .
Patrick SullivanAre there any like training programs or webinars or textbooks or anything that you ? How would you suggest someone like dip their toe in and see if there's a relevant piece for them ?
Justin KnoxYeah , I mean , that's a great question .
Justin KnoxAgain , I think it probably depends largely on , like , what type of learner you are and how you prefer to engage .
Justin KnoxI'm a person who's not afraid to say something stupid and like try something new , but meaning that I , you know , I won't know as much in that context .
Justin KnoxSo for me , what was most engaging and useful was to sign in and apply for these training programs and implementation science . You know , where you get involved in a group of people who are learning this , thinking about it , they connect you with experts and you really have a , you know , a chance to engage in discussion and react to things and talk through things , and that's the way that I learned best . You know , other people might prefer to sit down with a couple articles or an introductory textbook and really immerse themselves in the literature and then maybe be a little more , you know , refined in their thinking before they reach out , and that's probably totally fine too . But for me it's really about the collegiality , about the discussions , about , you know , working in teams and learning together and , you know , becoming part of a cohort and collaborating with mentors and , yeah , so for me it's much more social . But I think any way that works for you can get the job done .
Patrick SullivanOne thing I'll add to that is , I think , by its nature , unless you're already a provider like if you're already a medical care provider , he's working in an opioid substitution clinic or you know , like you already have the clinic the knowledge about the implementation environments . Implementation science by its nature requires some networking of folks who aren't our usual suspects but who have knowledge about the implementation settings . Because if you want to ask these questions about , I mean , you really almost need to go to the sleep apnea clinic or the HV prevention clinic or the TV , like whatever the thing is , and say , like if we brought this kind of intervention , like what would determine whether you could scale it up or not , like how would it look in your setting ? And so there is this piece of connection with how and where services are delivered that we that I usually know nothing about , and so that's also this like part of networking . So the methods get you to a certain distance , but by nature I think if you dive into this , you have to be willing to maybe cold call or maybe network a little bit and figure out how do we understand more about the settings in which this would be implemented , because there's a piece of humility here as implementation science scientists that we can understand , like the frameworks and collect the data , but in most cases we're not the providers or the service you know providers , and so that's an important perspective as well .
Patrick SullivanAnd thanks for mentioning that there's some training programs , some like online websites , that really describe some of these frameworks and , I think , talk about them in relatable ways , and we will put the links to the training programs and some of those major websites that you could check out in the show notes . So if you want to learn a little bit more , an easy way is just to visit one of you know re-aim or like one of the you know websites for a framework where they'll really walk through what the elements of implementation science frameworks are and give some examples , and that would be another great way to get oriented . That brings us to the end of this episode . Thanks again , Dr . Justin Knox , for joining us to talk about your article on the observational- implementation hybrid approach and especially for sharing a little bit more about your personal and professional background and how this paper came about . So pleasure to have you on the show .
Justin KnoxYeah , thanks so much . I really enjoyed the experience .
Patrick SullivanI'm your host , Patrick Sullivan . Thanks for tuning in to this episode and see you next time on EPI Talk: Behind the Paper . EPI Talk is brought to you by Annals of Epidemiology , the official journal of the American College of Epidemiology . For a transcript of this podcast or to read the article featured on this episode and more from the journal , you can visit us online at www . annals ofepidemiology . org .
