Code Gradients: Towards Automated Traceability of LLM-Generated Code

Large language models (LLMs) have recently seen huge growth in capability and usage. Within software engineering, LLMs are increasingly being used by developers to generate code. Code generated by an LLM can be seen essentially a continuous mapping from requirements to code. This represents a great opportunity within requirements engineering to use this mapping to provide traceability from requirements to LLM-generated code. The challenge is that the black-box nature of LLMs makes it difficult to trace requirements, while traditional approaches require extensive post-hoc testing or expert analysis. In this research preview, we explore the use of LLM explainability techniques to trace LLM-generated code back to requirements. By inspecting the gradients of LLM output, we develop a first attempt at tracing LLM inputs through to its generated code. We use this to estimate which low-level requirements have been met. Furthermore, through an automated iterative process, we re-query the LLM, instructing it to rewrite its code to meet the missing requirements. Our results suggest that the gradients of LLM outputs can be used to trace requirements through LLM code generation and that this traceability could potentially be used to improve generated code to better meet requirements. Future work is required to fully validate this result, but this represents a first step towards automatic traceability and verification of AI generated code.