Description

ctrueden

opened on Jul 26, 2024

Issue body actions

Requiring all Ops to be deterministic (same inputs produce the same output every time) excludes the possibility of wrapping probabilistic algorithms, which limits the scope of the Ops framework, and will detract from the value of the Ops index (#174). While I believe strongly in the importance of reproducibility, we can accommodate irreproducible-but-useful algorithms within Ops by adding a reproducible flag to the framework for each Op, false by default, which can be set when the Op author knows the algorithm will behave reproducibly. When this flag is set, the framework will know that outputs can be cached, which can be useful to improve time performance of repeated computation. And when the flag is unset, the framework can emit a warning explaining the problem to the user, thus raising awareness of the reproducibility concern.

This documentation and probably elsewhere as well will need adjustments to reflect this change.

Activity

1. ctrueden

   added

   enhancement

   on Jul 26, 2024
2. ctrueden

   added this to the unscheduled milestone on Jul 26, 2024
3. ctrueden

   added this to SciJava Ops Paper #2: Polygloton Jul 26, 2024
4. ctrueden

   moved this to Todo in SciJava Ops Paper #2: Polygloton Jul 26, 2024
5. gselzer

   self-assigned this

   on Jul 31, 2024
6. gselzer commented on Jul 31, 2024

   

   gselzer

   on Jul 31, 2024

   Member

   More actions

   I'm actually really in favor of this. As reproducibility is something that cannot actually be guaranteed within each Op, the SciJava Ops application-wide guarantee was always going to be an aspiration that we cannot be sure about.

   I was initially disapproving of the false-by-default reproducibility hint, however of course it's necessary, as you cannot be sure that an Op is reproducible - it's best to evaluate the claim and then add the hint when we have validated reproducibility.

   I believe that this change will help us focus on our goals of accessibility, and of course a hint will enable reproducibility for those who want it.

   When this flag is set, the framework will know that outputs can be cached, which can be useful to improve time performance of repeated computation.

   Yeah, this is currently not supported, but a WeakHashMap within the OpWrapper (example: here) would make checking simple and space-efficient.

   Reacted by Curtis Rueden
7. gselzer commented on Aug 8, 2024

   

   gselzer

   on Aug 8, 2024

   Member

   More actions

   #261 edits the spots in the docs that I could find - is there anything else we need here, @ctrueden?

Issue actions