Update LICENSE and TODOs in README (#14)
* Update LICENSE with our copyright notice * Update README.md * fix readme anchor * Update README.md
This commit is contained in:
parent
7cebaba8e4
commit
e3b4b85caa
2 changed files with 12 additions and 10 deletions
1
LICENSE
1
LICENSE
|
@ -1,6 +1,7 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2023 Georgi Gerganov
|
||||
Copyright (c) 2023 SJTU-IPADS
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
|
19
README.md
19
README.md
|
@ -54,13 +54,18 @@ We have tested PowerInfer on the following platforms:
|
|||
- x86-64 CPU and NVIDIA GPU on Linux
|
||||
- Apple M Chips on macOS (As we do not optimize for Mac, the performance improvement is not significant now.)
|
||||
|
||||
And new features coming soon:
|
||||
|
||||
- Mistral-7B model
|
||||
- Online fine-grained FFN offloading to GPU
|
||||
- Metal backend for sparse inference on macOS
|
||||
|
||||
## Getting Started
|
||||
|
||||
- [Installation](##setup--installation)
|
||||
- [Model Weights](##model-weights)
|
||||
- [Installation](#setup-and-installation)
|
||||
- [Model Weights](#model-weights)
|
||||
|
||||
## Setup & Installation
|
||||
## Setup and Installation
|
||||
### Get the Code
|
||||
|
||||
```bash
|
||||
|
@ -70,12 +75,7 @@ cd PowerInfer
|
|||
### Build
|
||||
In order to build PowerInfer you have two different options. These commands are supposed to be run from the root directory of the project.
|
||||
|
||||
Using `make` on Linux or macOS:
|
||||
```bash
|
||||
make
|
||||
```
|
||||
|
||||
Using `CMake`:
|
||||
Using `CMake` on Linux or macOS:
|
||||
* If you have one GPU:
|
||||
```bash
|
||||
cmake -S . -B build -DLLAMA_CUBLAS=ON
|
||||
|
@ -130,6 +130,7 @@ PowerInfer achieves up to 11x and 8x speedup for FP16 and INT4 models!
|
|||
We will release the code and data in the following order, please stay tuned!
|
||||
|
||||
- [x] Release core code of PowerInfer, supporting Llama-2, Falcon-40B.
|
||||
- [ ] Support Mistral-7B
|
||||
- [ ] Release perplexity evaluation code
|
||||
- [ ] Support Metal for Mac
|
||||
- [ ] Release code for OPT models
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue