mirrors/cosmopolitan
1
0
Fork 0
mirror of https://github.com/jart/cosmopolitan.git synced 2025-09-10 02:33:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Introduce prompt caching so prompts load instantly

Browse source 
This change also introduces an ephemeral status line in non-verbose mode
to display a load percentage status when slow operations are happening.
This commit is contained in:
Justine Tunney 2023-04-28 16:15:26 -07:00
parent bf6459e324
commit b31ba86ace
No known key found for this signature in database
GPG key ID: BE714B4575D6E328
7 changed files with 333 additions and 103 deletions
Download patch file Download diff file Expand all files Collapse all files

2
third_party/ggml/README.cosmo vendored
View file

@ -16,7 +16,9 @@ ORIGIN
LOCAL CHANGES
- Make it possible for loaded prompts to be cached to disk
- Introduce -v and --verbose flags
- Reduce batch size from 512 to 32
- Don't print stats / diagnostics unless -v is passed
- Reduce --top_p default from 0.95 to 0.70
- Change --reverse-prompt to no longer imply --interactive