How has DS & ML helped in observing systems under test to understand and predict their behavior at scale?

Incorporating data science has helped us see patterns in the data we haven't seen before, and understanding how the system reacts is crucial to improving it, in other words, information is power. Moreover, DS helps us analyze a plethora of data that spans multiple years of testing, which means we can not only see the trends but also design ML models that can pick up on these trends, with a speed, accuracy, and thoroughness that cannot be achieved "the old fashioned way". Two things to have when working on a DS/ML-related project are knowledge and research. There are so many different methods of learning and statistical analysis. Knowing which ones are the suitable solution for a specific problem and how to implement it in an efficient way comes from a combination of education and hands-on experience, which is what makes a degree in this field valuable. It's also important to be constantly on the lookout for new advancements. This is a field with unprecedented growth, so it's best to be constantly on the lookout for what's new and what has been tried before.

Thank you for the detailed answer! Knowing available statistical methods available and knowing which one to use for a particular DS/ML situation seems to be critical. In your experience, how would learning these statistical methods in a college setting vs separately (e.g. Coursera) have affected your ability to apply them effectively in your day-to-day job?

Even though having an academic background in these fields matters, education cannot be decoupled from online resources. What a theoretical approach brings to the table is the ability to dissect the problem, rather than jumping to conclusions. Because data science is mostly about understanding your data, and then understanding the methods. The implementation is almost trivial after a certain level of expertise, but unfortunately many online courses focus on that. Hence, it's important to find resources that can give you analytical skills.