written by Eric J. Ma on 2021-09-30 | tags: data science career faq
Q: How was my transition from the academic world to the industry?
I found the transition relatively easy because, I think, of the environment that I first joined. The Novartis Institutes for BioMedical Research, NIBR, was famed for leaning towards the academic side back in 2017. Thus, over the 3.5 years that I was there, I was given free rein to decide what to work on. The operational freedom was quite important to me as I had experienced a similar environment in graduate school.
That lesson on the environment turned out to be more critical than I appreciated early on. We might live in an individualistic society, yet we're more affected by our community and environment than we might expect. (One might even make the case that a phase change might happen when a community collectively turns individualistic... that's an irony to explore in another setting.) During my time there, I found NIBR's environment to be pretty forgiving, thanks to its slower pace than other companies. Generally, my colleagues were kind and intelligent (in that order). Colleagues at the same level were idealists who strived to do good science. Having that community of similarly skilled and motivated colleagues was essential to me early in my career.
Q: What essential steps/skills that I gained allowed me to advance in the field?
First off, we should get one point out of the way - I don't consider promotions my primary indicator of advancement. For me, it is a side-effect of doing good work. What profoundly satisfies me the most at work is that the projects I work on accelerate science to the speed of thought. I hold the conviction that if I'm advancing the field of science that way, I'll eventually share in the team's success.
That perhaps is the most critical lesson that others way more experienced than me have given to me: to stay mission-focused and to not think much about promotions, salary, and more, for they come, eventually. To transcend the material and not think much of credit gives us the freedom to pursue high-value work in the service of others. The irony here is that when you do high-value work in the service of others, you'll then have the credibility points to ask for a high-value, market-rate compensation for that work.
So the most important thing I see is finding, buying into, or creating a transcendent, high-value, service-oriented mission for yourself. Then, use your powers of logic to deduce the high-value actions you need to take to serve others. All of business, science, and life, in general, should be about deep service to others. If we start focusing inwards on our own desires, we'll end up deeply unsatisfied. But, on the other hand, I firmly believe that if we keep service to others squarely in our sights, we'll always find unique and fun work to do.
Now, everything I said above does not exclude looking around and benchmarking whether I'm being treated fairly or not. Still, it's important to note the order of priorities: I am first concerned with the transcendental mission and then worried about making sure I'm being treated fairly. With that order of importance, I can be free to do my best work first. Doing so lets me accumulate credibility points to advocate for fair treatment of myself and others around me. Flip that order of priority, and I am sure I will be treated as a pariah with no credibility. So our credibility points must be earned first.
As for the tangibles, such as salary and title, some of my following observations could be of help:
Okay, so beyond the overarching "how do you advance" point, I'd like to address "how to do good work." I'm trusting that you probably know a lot of these points anyway. So I'm expecting that these points should serve merely as reminders rather than advice. I'll explain this in the context of my idealized project lifecycle.
Firstly, when you embark on a project, I would encourage you to sit down and carefully write out a two-page (~1000 word) document for yourself. This document should provide an overview of the motivation, needs, and possible solutions for a problem your colleague brings to you or that you have identified. This document is akin to a project charter. It's very tempting to dive head-first into programming as that feels like our currency of data science work. But the actual currency of data science work is our insights; code is merely our medium to get there. Writing a document that accurately, concisely, and clearly communicates an overview of the project can help you design the solution; sometimes, it also kickstarts the flow of creative ideas. In other settings, it might illuminate the most straightforward method that solves the problem. Clarity and empathy, accurately communicated, are what you need to bring to the table for your insights to be accepted. Without that, your colleagues will give you "the eye" and wonder what you're talking about.
Secondly, once you've spent the time wrestling with the problem, it's time to design and execute the solution. This is the project phase where we will end up doing programming work. In choosing our strategy, the standard advice circulating around is to "pick the simplest solution." However, I think that is only partially useful as an answer. A more helpful response, in my opinion, is to pick the principled solution that solves the problem at its core. Looking beyond the fluff of most issues, you can probably identify a core problem that needs to be solved. This is the problem you need to tackle first. The complexity involved in a problem usually comes from edge cases, h uman-made exceptions introduced to other human-made reasons (e.g. legal or ethical). Those can be composed later but should not affect your solution core at the beginning.
Thirdly, iterate with regular communications with colleagues. Junior data scientists may find it very tempting to dive into a hole and not surface for a good month or so, with the positive intent of delightfully surprising your colleagues at the end of your adventure. The reality is quite different, though:
As I've learned, it's often non-trivial to arrive at a simple solution that offers you high leverage over a problem. Regularly communicating the challenge to your colleagues can help them better appreciate your work.
And in case you mistakenly think I communicate perfectly, I don't. That paragraph above is as much a reminder for myself as it is advice to share, too.
Finally, you'll want to think about how to round out a project. One useful outlet is a final presentation - a debrief for your main stakeholder(s). Try to schedule something, even if its only purpose is to ceremonially mark the end of a project. The goal, at the minimum, is to bring closure to a project. Whether or not the stated purposes of a project were accomplished, celebrate everything learned through the process. Having a clear boundary at the end of a project brings a lot of clarity to everyone at the table. You can also leverage the final presentation to shape collective memory of what was accomplished and done - which can mean credibility points for you to spend on other projects. Being able to schedule a "final" presentation means that you'll also be less likely to be bogged down by a constant trickle of follow-up requests. Heed the words of one who has experienced it before!
Q: What tips do you have for conveying technical points to a non-technical audience?
For this, I have taken a lot of inspiration from quotes by Richard Feynman, particularly w.r.t. learning new things. He has this quote along the lines of being able to explain things to a five-year-old. The broader point is explaining a technical idea at multiple levels of detail and adapting to what your audience is looking for.
Conveying technical points to a non-technical audience always takes time and lots of preparation. Therefore, doing a project well will necessarily mean also communicating it well. For this reason, I found that if I want to do my projects well, I need to stop willy-nilly accepting new requests to do projects. Rather, I need to focus on delivering well on one or two of them.
Q: What tips do you have for contributing to the community, like writing blog posts and such?
Share what you learn, and that's good enough. Treat the blog post as your learning log, don't worry about impressing others. I found that when I worry about impressing others, I end up not writing. That ironically hurts my ability to learn. So forget about impressing others and simply focus on recording what you've learned.
I send out a newsletter with tips and tools for data scientists. Come check it out at Substack.
If you would like to receive deeper, in-depth content as an early subscriber, come support me on Patreon!