deterministic behaviour CellTypeR

[pedriniedoardo]

Hello, thank you for developing the tool.
On the site, you highlight that the method is deterministic (same input -> same output).

I have a test run ~2h apart using the same input and same parameters. I do not get "exactly" the same result.
Test 7:44 PM

Test 5:01 PM

I was wondering if this is expected or not.

[suu-yi]

Hello @pedriniedoardo!
We have a similar issue open in the main CyteType, which you might be interested in following the conversation there:
NygenAnalytics/CyteType#80
Hope this helps!

[pedriniedoardo]

I see, thank you for clarifying @suu-yi .
To summarise, while you apply some solutions to make it deterministic (0 temperature and convergence using multiple review rounds), some randomness is still expected due to the LLM-backend nature. Nonetheless, the "semantic" annotation should be similar in every run, like I should not see a lymphocyte cluster to be called fibroblast from run-to-run, while I could see Naive CD4+ T cells being labelled as Central Memory CD4+ T cells from run-to-run (like it has happened in my case).
You are planning to implement a macro-grouping feature to further mitigate run-to-run variation. But this has not been implemented yet for both the Python and R implementations.
Is that correct?