-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
Proposal summary
I have several prompts that I want to reuse, ex. I may want to see if it works better in one prompt or batch several prompts together. The fact that it would be versioned and work within opik is great. The workaround would be to dump it into a dataset and work from there, but then I lose the versioning and would have to manage a field within the dataset continuously.
So the goal would be having something like:
Prompt library:
Prompt A: does this person like cheese
Pompt B: do they like puppies
Prompt C: do they like toyotas
And then I can either send all messages separately so
user message: {{prompt A}}
user message: {{prompt B}}
user message: {{prompt C}}
Or I can send them all at once
user message: {{Prompt A}} {{Prompt B}} {{Prompt C}}
With the goal of being able to run LLM as judge or human judge etc and seeing which performs better
Motivation
You can get a lot from this, faster results, lower cost, etc. Also keeps everything organized.