컨텍스트 캐싱
예 1 : 긴 텍스트 Q & A
messages: [
{"role": "system", "content": "You are an experienced financial report analyst..."}
{"role": "user", "content": "<financial report content>\n\nPlease summarize the key information of this financial report."}
]messages: [
{"role": "system", "content": "You are an experienced financial report analyst..."}
{"role": "user", "content": "<financial report content>\n\nPlease analyze the profitability of this financial report."}
]user 메시지에서 system 메시지 + <financial report content> 입니다. 두 번째 요청 에서이 접두사 부분은 "캐시 히트"로 계산됩니다.예제 2 : 다소 라운드 대화
messages: [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "What is the capital of China?"}
]messages: [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "What is the capital of China?"},
{"role": "assistant", "content": "The capital of China is Beijing."},
{"role": "user", "content": "What is the capital of the United States?"}
]system 메시지와 user 메시지를 재사용 할 수 있으며, 이는 "캐시 히트"로 계산됩니다.예 3 : 소수의 샷 학습 사용
messages: [
{"role": "system", "content": "You are a history expert. The user will provide a series of questions, and your answers should be concise and start with `Answer:`"},
{"role": "user", "content": "In what year did Qin Shi Huang unify the six states?"},
{"role": "assistant", "content": "Answer: 221 BC"},
{"role": "user", "content": "Who was the founder of the Han Dynasty?"},
{"role": "assistant", "content": "Answer: Liu Bang"},
{"role": "user", "content": "Who was the last emperor of the Tang Dynasty?"},
{"role": "assistant", "content": "Answer: Li Zhu"},
{"role": "user", "content": "Who was the founding emperor of the Ming Dynasty?"},
{"role": "assistant", "content": "Answer: Zhu Yuanzhang"},
{"role": "user", "content": "Who was the founding emperor of the Qing Dynasty?"}
]messages: [
{"role": "system", "content": "You are a history expert. The user will provide a series of questions, and your answers should be concise and start with `Answer:`"},
{"role": "user", "content": "In what year did Qin Shi Huang unify the six states?"},
{"role": "assistant", "content": "Answer: 221 BC"},
{"role": "user", "content": "Who was the founder of the Han Dynasty?"},
{"role": "assistant", "content": "Answer: Liu Bang"},
{"role": "user", "content": "Who was the last emperor of the Tang Dynasty?"},
{"role": "assistant", "content": "Answer: Li Zhu"},
{"role": "user", "content": "Who was the founding emperor of the Ming Dynasty?"},
{"role": "assistant", "content": "Answer: Zhu Yuanzhang"},
{"role": "user", "content": "When did the Shang Dynasty fall?"},
]캐시 적중 상태 확인
usage 섹션에 두 개의 필드를 추가하여 요청의 캐시 적중 상태를 반영했습니다.1.
2.
하드 디스크 캐시 및 출력 임의성
추가 메모
1.
2.
3.