To me this is the most interesting llama related project right now:
https://github.com/ggerganov/llama.cpp
Very easy to setup:
1. git clone the project and setup it
https://github.com/ggerganov/llama.cpp#get-the-code
2. Select your LLaMA2 model already quantizated from here: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main
3. ./main -m ./models/your_downloaded_model.bin -n 128
DesobedienteTecnológico 🛠️⚡
npub1dt…pgte6
2023-08-01 20:25:35
in reply to nevent1q…lpql
Author Public Key
npub1dtmp3wrkyqafghjgwyk88mxvulfncc9lg6ppv4laet5cun66jtwqqpgte6Published at
2023-08-01 20:25:35Event JSON
{
"id": "cb2eea547e23ec067f1e25ac5c8f5973eee7a614b16ba25351e158815ffaf23d",
"pubkey": "6af618b876203a945e48712c73eccce7d33c60bf46821657fdcae98e4f5a92dc",
"created_at": 1690914335,
"kind": 1,
"tags": [
[
"e",
"d8729ee5395021426c5eb2b7e4537b6f394dd520ff7dc0dae7fb0b419df88eff",
"",
"reply"
],
[
"p",
"24e37c1e5b0c8ba8dde2754bcffc63b5b299f8064f8fb928bcf315b9c4965f3b"
],
[
"r",
"https://github.com/ggerganov/llama.cpp"
],
[
"r",
"https://github.com/ggerganov/llama.cpp#get-the-code"
],
[
"r",
"https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main"
]
],
"content": "To me this is the most interesting llama related project right now:\nhttps://github.com/ggerganov/llama.cpp\n\nVery easy to setup:\n\n1. git clone the project and setup it\nhttps://github.com/ggerganov/llama.cpp#get-the-code\n2. Select your LLaMA2 model already quantizated from here: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main\n3. ./main -m ./models/your_downloaded_model.bin -n 128\n\n",
"sig": "aa719309c596e9db56b778be82a6f164edd28c5e4be4202a6a5ebbeaa303878c0e92763beebbd2fa8e09405b6db31f10ab2f2b8d72a1915193c16cd439aede3f"
}