Assessing the Reasoning Capabilities of Large Language Models for Scenario Analysis in Product Development

Understanding scenario-specific requirements is critical for early-stage product development as it enables structured exploration of potential use-cases, risks, and system constraints for the design process. Scenario analysis is a foresight method in which the entire product lifecycle is systematically considered from development to disposal, capturing interactions between the product and its environment and how the user anticipates potential challenges of design implications. With large language models (LLMs), the interest in leveraging application capabilities to retrieve specific information from large-scale unspecific stakeholder scenarios is growing. However, the extent to which LLMs can understand scenarios in depth remains underexplored for defined product development tasks. For this reason, this paper conducts a potential analysis of whether and to what extent LLMs are capable of deriving ISO-compliant requirements from vague formulations. Our use case focuses on the challenge of achieving design requirements for an “unproblematic transport of a motorcycle using a trailer.” This vague formulation requires both technical reasoning (e.g., load distribution, safety constraints) and creativity (e.g., anticipating edge cases). Using a Nebius API key, we test three sizes of the LLaMA model with 8 billion, 70 billion, and 405 billion parameters while keeping the system code and the data pipeline constant. We evaluate the model variations across multiple dimensions: scenario comprehension, technical feasibility, and reasoning depth. This approach allows a direct comparison of training parameter influence on reasoning performance in the application of scenario analysis using one-shot prompting. This investigation contributes to advancing AI-supported technical scenario analysis in product development by offering a methodology for systematically benchmarking LLMs in an applied engineering context.