minus-squareHappy_Chicken9835@alien.topBtoLocalLLaMA•Deploy LLaMA 2 on AWS: What's the best option?linkfedilinkEnglisharrow-up1·1 year agoI deployed Llama 2 (GGUF and using CPU) as an Amazon ECS fargate service I just bundled my entire Docker build into ECR and fired up my container linkfedilink
minus-squareHappy_Chicken9835@alien.topBtoLocalLLaMA•ORCA 2 Released open source!linkfedilinkEnglisharrow-up1·1 year agoThe bloke has a few quantized variants A gguf 7B: https://huggingface.co/TheBloke/Orca-2-7B-GGUF linkfedilink
I deployed Llama 2 (GGUF and using CPU) as an Amazon ECS fargate service
I just bundled my entire Docker build into ECR and fired up my container