Looking for a paper that I vaguely remember having read the abstract of recently.
As I recall it, it showed that given black-box access to a (typical generative auto-regressive) #LLM #AI 'alpha' that cost some amount $C to train, you can make a new LLM AI 'beta' whose effectiveness is (in some significant sense) nearly as good as the effectiveness of 'alpha', and that costs significantly less than $C to train.
Is that a thing? Search terms like "learning transfer" find things that are vaguely related, but not quite that.
Turns out, I think, that I was thinking of the #Alpaca paper, in which (somewhat more subtly) they show improving the (leaked) #LLaMA 7B model to be close in effectiveness to the GPT-3.5 model, by using black-box API access to the GPT-3.5 model, for just US$500 or so, which is presumably significantly less than the cost-delta between LLaMA 7B and GPT-3.5.
At least that's how I interpret the Alpaca paper ( https://crfm.stanford.edu/2023/03/13/alpaca.html ) and I think that's the thing I was remembering...
