Add k/v caching for autoregressive generation

This commit is contained in:
Zygimantas Straznickas
2023-04-20 18:39:14 -07:00
parent 874af1bae9
commit 15606ed12f
4 changed files with 94 additions and 30 deletions

2
.gitignore vendored
View File

@@ -1,2 +1,2 @@
__pycache__/
.venv