vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add LLAMA_DEFAULT_RMS_EPS so we can change the default (#2384)

Browse source 
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow 2023-07-25 18:35:53 +03:00 • committed by GitHub
parent 07aaa0f63f
commit eb542d3932
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 13 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

6
examples/baby-llama/baby-llama.cpp
View file

@ -8,7 +8,11 @@
#pragma warning(disable: 4244 4267) // possible loss of data
#endif
static const float rms_norm_eps = 1e-6f;
#ifdef LLAMA_DEFAULT_RMS_EPS
static const float rms_norm_eps = LLAMA_DEFAULT_RMS_EPS;
#else
static const float rms_norm_eps = 5e-6f;
#endif
float frand() {
return (float)rand()/(float)RAND_MAX;