Field Experimental Evidence of the Effects of AI on Knowledge Worker

In a study conducted with Boston Consulting Group, researchers examined the impact of Artificial Intelligence (AI) on complex tasks performed by knowledge workers. In the experiment, 758 consultants were randomly assigned to three groups: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview. The results showed that tasks within the capabilities of AI were completed more efficiently and with higher quality when AI was used. Consultants using AI completed 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced results of over 40% higher quality compared to the control group. Both consultants below and above the average performance threshold benefited significantly from AI augmentation, with increases of 43% and 17% respectively. However, when faced with a task outside the current capabilities of AI, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. Additionally, the analysis revealed two distinct patterns of successful AI use: "Centaurs" who divided their solution-creation activities between themselves and AI, and "Cyborgs" who fully integrated their task flow with AI and had continuous interaction with the technology. Overall, the study demonstrates the potential benefits and limitations of AI in enhancing the productivity and quality of knowledge-intensive tasks.