You can use vicuna-7b-1.1. No need for chat prompts. Just slam in your data and end it off like so
Generate a JSON with this and that
{"this": "
Lower the temperature to minimum for deterministic results, fine tune the other parameters if needed. And have a stop token for JSON closing tag like so }.
That usually works perfectly fine for me in most scenarios. Best: that stuff runs on RTX 3080 with 15token/s (quite fast!). Also vicuna-7b is pretty much as good as gpt-3 when it came out.
Vicuna-7b is much better than Gpt4All, but still struggles with math - I can't wait until my new work computer comes in, I will try to run the new StableLM models
Generate a JSON with this and that {"this": "
Lower the temperature to minimum for deterministic results, fine tune the other parameters if needed. And have a stop token for JSON closing tag like so }.
That usually works perfectly fine for me in most scenarios. Best: that stuff runs on RTX 3080 with 15token/s (quite fast!). Also vicuna-7b is pretty much as good as gpt-3 when it came out.