LLMs regurgitate a version of whatever they were trained on. If you train them on typically buggy code, that suggests you’ll get back typically buggy code. In a new preprint, “LLMs are …