Hiring data and machine learning (ML) engineers is certainly on the rise, here at DevEngine we’ve witnessed a significant uptick in job postings in the last few months. A closer look reveals that these opportunities are predominantly in the US and tend to come from larger enterprises. This trend underscores a broader narrative: big companies are the more eager adopters of advanced AI and data science technologies, while medium-sized and smaller businesses seem to be taking a more cautious approach.
This observation sets the stage for a broader conversation we had at Microsoft’s ‘AI in Action’ event series last week, an invite we were thrilled to receive from Vancouver Tech Journal. The event assembled a thought-provoking panel, including Hon. Brenda Bailey, BC’s Minister of Jobs, Economic Development, and Innovation; David Seymour, VP and GM of Microsoft Vancouver; and Walter Pela, Managing Partner of KPMG Vancouver. The event wasn’t just a local affair. Although it focused on British Columbia, the insights shared were gold for anyone looking to harness AI’s potential worldwide. The discussion was an eye-opener on AI’s impact not just in tech circles but across businesses and governments.
Here’s a takeaway that stuck with us (paraphrasing a bit here): If your business isn’t collecting data with an eye towards AI right now, as of February 2024, you’re already a step behind some of your competitors. It’s clear that having a stash of your own data to train AI models is crucial. This isn’t about keeping up; it’s about leading with AI-driven insights that can transform your business.
Even if it’s uncertain what the AI application or LLM will be two to three years from now, it will most certainly require a private data set and the time to get on that bandwagon is… yesterday. Begin collecting the data sets that your business uniquely has. You can then train expert models that are a lot more valuable to your business, customers, and processes than some public model you’ve faced before. What lies beyond that is applying optimization at scale.
Here’s the thing – it’s not just about having data. It’s about what you do with it. Forward-thinking companies across all industries are leveraging data science and machine learning (DS/ML) not just for growth but to fundamentally enhance how they operate and serve their customers. For smaller companies watching from the sidelines, this serves as a wake-up call. The path to leveraging AI effectively involves not just data collection but investing in the right talent and tools. As the discussion revealed, data science and machine learning are no longer optional but critical for driving growth, enhancing customer experiences, and improving predictability.
If you are actively pursuing AI projects or ready to take action, take a look at the “State of Data + AI” report from Databricks to wrap your head around DS/ML trends and tools. This report brings to light some compelling stats:
• The use of SaaS LLM APIs (to access services like ChatGPT) ballooned by 1310% from November 2022 to May 2023.
• There’s a 411% year-over-year increase in models being rolled out into production.
• The efficiency in model experimentation is on the rise – for every three models, one makes it to production.
• Beyond BI, the growth in advanced data use cases shows companies are pushing the boundaries of what’s possible.
• The most rapid growth? Dbt, with a 206% increase in customer count by year.
These trends underscore the critical need for businesses to hire data engineers and machine learning engineers. As companies strive to position themselves at the forefront of innovation, the demand for skilled professionals capable of navigating the complexities of data infrastructure and developing advanced ML models has never been higher.
For organizations looking to lead in the next generation of data-driven industries, the message is clear: investing in the right talent to harness the power of DS/ML is not just an option, but a necessity. Whether it’s to refine your data collection strategies, train bespoke AI models, or simply keep pace with the rapid advancements in technology, the time to act is now.