Can AI fully automate the research process?

Researchers at ByteDance have released DAComp, a benchmark created to test how well AI can manage the full analytics lifecycle — from data processing to insight and recommendation. And they've looked at how well state-of-the art models perform on these benchmarks today.⁣
⁣
𝐓𝐋𝐃𝐑: AI is now an OK analyst (for less complex tasks) and a terrible strategist.⁣
⁣
Their paper looks at AI performance across two dimensions:⁣
⁣
(𝟏) 𝐓𝐡𝐞 “𝐡𝐚𝐫𝐝” 𝐝𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧 (𝐭𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥 𝐞𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧)⁣
⁣💻 This includes breaking down a business question, planning analyses, writing queries, running calculations, and producing charts.⁣ AI can handle many of these steps, but performance is uneven — it does well on straightforward descriptive tasks, yet reliability drops when logic becomes layered or when multiple analytical steps must be sequenced correctly.⁣
⁣
(𝟐) 𝐓𝐡𝐞 “𝐬𝐨𝐟𝐭” 𝐝𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧 (𝐚𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐚𝐥 𝐫𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 & 𝐢𝐧𝐭𝐞𝐫𝐩𝐫𝐞𝐭𝐚𝐭𝐢𝐨𝐧)⁣
⁣🧠 This involves interpreting results, identifying drivers, weighing explanations, and turning findings into clear recommendations.⁣ Here, AI still struggles: it can summarise outputs, but often cannot deliver grounded diagnosis or meaningful strategic direction.⁣
⁣
My take-away is that AI accelerates parts of the analytical workflow, but its limitations grow as tasks become more complex and interpretive, and human expertise remains essential for context, judgment, and synthesis.⁣
⁣
Good initiative to create a dedicated benchmark for this.⁣

Link to paper:https://arxiv.org/pdf/2512.04324