Hello,
I got the error "Found invalid prompt (length > 256) for behavior" for one of my submissions. Is there a limit on the number of character/tokens that can be in the prompt?
Posted by: iamgroot42 @ Aug. 11, 2023, 7:58 p.m.Hello,
Yes, submitted test cases must be no more than 256 tokens long after tokenization. Given a test case, the generation is also limited to at most 512 new tokens. This limit is for reducing the costs of evaluation and generating submissions. It has been in effect since the beginning of the competition, but we forgot to include it in the competition website. My apologies for this oversight. We now mention it in the website, CodaLab pages, and starter kit. Thank you for bringing this to our attention.
All the best,
Mantas (TDC co-organizer)