prompt injection
Large Language Models can be backdoored by introducing just a limited number of “poisoned” documents during their training, a team of researchers from the UK’s Alan Turing Institute and partners found. “Injecting backdoors through data poisoning may be easier for large models than previously believed as the number of
Tenable researchers able to circumvent Google's AI security with successful prompt injections.
Prompt injections to break safeguards on widely available LLMs meanwhile are also widely available.
"When we stumbled upon this library we immediately thought that connecting an LLM to SQL query execution could result in a disastrous SQL injection..."