This presentation examines human-AI competence in cyber-physical systems, focusing on AI-enabled maritime operations. In a new project led by Caitlin Bentley (King's College) and Gordon Meadow (Seabot Maritime), the presentation addresses two key risks: operator deskilling as roles shift from active operation to passive monitoring, and workforce development pathways amid skills shortages. It will explore the impact of simulated training for unexpected failures and questions surrounding operator qualifications. The aim is to ensure that autonomous shipping augments human capabilities rather than replaces them.