Prompt injection - Revision history

Rice: /* Attacks */

2024-05-23T20:58:18Z

Attacks

← Older revision		Revision as of 20:58, 23 May 2024
Line 15:		Line 15:
	* Escape separation attacks "adds space" between system and user		* Escape separation attacks "adds space" between system and user
	* Completion attacks fakes a response to trick the model into thinking a new query is beginning.		* Completion attacks fakes a response to trick the model into thinking a new query is beginning.
			** Completion real attacks adds real delimiters to fake it
			** Completion close attacks adds adjacent delimiters
			** Completion other uses other delimiters.

	= Defense strategies =		= Defense strategies =

Rice: /* Attacks */

2024-05-23T20:57:00Z

Attacks

← Older revision		Revision as of 20:57, 23 May 2024
Line 14:		Line 14:
	* Escape character attacks "deletes" previous instructions		* Escape character attacks "deletes" previous instructions
	* Escape separation attacks "adds space" between system and user		* Escape separation attacks "adds space" between system and user
			* Completion attacks fakes a response to trick the model into thinking a new query is beginning.

	= Defense strategies =		= Defense strategies =

Rice: /* Attacks */

2024-05-23T20:56:08Z

Attacks

← Older revision		Revision as of 20:56, 23 May 2024
Line 13:		Line 13:
	* Ignore attacks tells the model to ignore previous instructions		* Ignore attacks tells the model to ignore previous instructions
	* Escape character attacks "deletes" previous instructions		* Escape character attacks "deletes" previous instructions
			* Escape separation attacks "adds space" between system and user

	= Defense strategies =		= Defense strategies =

Rice: /* Vulnerability */

2024-05-23T20:55:22Z

Vulnerability

← Older revision		Revision as of 20:55, 23 May 2024
Line 8:		Line 8:

	Prompt injection exploits the single-channel nature of LLM's, where user prompts and system prompts are simply concatenated together and processed.		Prompt injection exploits the single-channel nature of LLM's, where user prompts and system prompts are simply concatenated together and processed.

			= Attacks =
			* Naive attack simply inject additional instruction
			* Ignore attacks tells the model to ignore previous instructions
			* Escape character attacks "deletes" previous instructions

	= Defense strategies =		= Defense strategies =

Rice at 20:47, 23 May 2024

2024-05-23T20:47:41Z

← Older revision		Revision as of 20:47, 23 May 2024
Line 1:		Line 1:
	[[Category:~~Cybersecurity~~]]		[[Category:LLM security]]

	A '''prompt [[injection]]''' attack involves a user injecting a malicious		A '''prompt [[injection]]''' attack involves a user injecting a malicious
Line 12:		Line 12:
	* [[StruQ]] rejects all user instructions		* [[StruQ]] rejects all user instructions
	* [[Instruction hierarchy]] rejects user instructions that are misaligned with the system prompt		* [[Instruction hierarchy]] rejects user instructions that are misaligned with the system prompt

			= Difference from jailbreaking =

			[[Jailbreak]] is a similar but different classification of attacks. Instead of attacking the application, it targets the model to produce inappropriate output.

Rice at 20:28, 23 May 2024

2024-05-23T20:28:45Z

← Older revision		Revision as of 20:28, 23 May 2024
Line 2:		Line 2:

	A '''prompt [[injection]]''' attack involves a user injecting a malicious		A '''prompt [[injection]]''' attack involves a user injecting a malicious
	instruction in an LLM-integrated application, in which user input was		instruction in an [[LLM-integrated application]], in which user input was
	intended to act as only data.		intended to act as only data.

			= Vulnerability =

			Prompt injection exploits the single-channel nature of LLM's, where user prompts and system prompts are simply concatenated together and processed.

	= Defense strategies =		= Defense strategies =
	* [[StruQ]] rejects all user instructions		* [[StruQ]] rejects all user instructions
	* [[Instruction hierarchy]] rejects user instructions that are misaligned with the system prompt		* [[Instruction hierarchy]] rejects user instructions that are misaligned with the system prompt

Rice at 20:25, 23 May 2024

2024-05-23T20:25:17Z

← Older revision		Revision as of 20:25, 23 May 2024
Line 4:		Line 4:
	instruction in an LLM-integrated application, in which user input was		instruction in an LLM-integrated application, in which user input was
	intended to act as only data.		intended to act as only data.

			= Defense strategies =
			* [[StruQ]] rejects all user instructions
			* [[Instruction hierarchy]] rejects user instructions that are misaligned with the system prompt

Rice: Created page with "Category:Cybersecurity A '''prompt injection''' attack involves a user injecting a malicious instruction in an LLM-integrated application, in which user input was intended to act as only data."

2024-05-23T20:23:54Z

Created page with "Category:Cybersecurity A '''prompt injection''' attack involves a user injecting a malicious instruction in an LLM-integrated application, in which user input was intended to act as only data."

New page

[[Category:Cybersecurity]]

A '''prompt [[injection]]''' attack involves a user injecting a malicious
instruction in an LLM-integrated application, in which user input was
intended to act as only data.