Hi,
I tried to apply SERAC to our constructed datasets in MLLM, but the performance turned out to be very low, with a rewrite accuracy of around 0.1.
It tends to output the original answers, which suggests that the edits were not applied successfully. Since I did not modify the model or the method, I am wondering if you could share any insights or suggestions based on your experience with this approach.
Thanks!
Hi,
I tried to apply SERAC to our constructed datasets in MLLM, but the performance turned out to be very low, with a rewrite accuracy of around 0.1.
It tends to output the original answers, which suggests that the edits were not applied successfully. Since I did not modify the model or the method, I am wondering if you could share any insights or suggestions based on your experience with this approach.
Thanks!