Dear organizers,
I have a question about a starting kit.
i'm currently working on track1(videos) and it looks like the files of ground-truth trajectories (trajs_fov_0) are not match to the particle's positions of videos.
the FOVs have 128x128 pixels but the x,y positions of trajectories have higher values than 128.
Is this an error of trajs_fov_.csv files?
Hi, thanks for your question. The trajectories coordinates in the csv have absolute units w.r.t. to the full field of simulation, which is much larger than the FOV of the video. If you want to match the csv coordinates with positions in the FOV, you can use the coordinate at time 0 of one of the VIP. It is basically a change of the origin of the reference system.
Posted by: gorka.munoz @ March 8, 2024, 10:15 a.m.Dear organizer,
I'd like to know what is the evaluation method of JSC and RMSE of TP in the case when there is no changepoints (not only in the single-state scenario),
i expected 1 / (1 + FP + FN ) for JSC but it doesn't seem so, i couldn't find the metric in this case on the paper.
Thank you so much in advance.
Best
Junwoo
Hi Gorka,
If i put an additional FP in a single-state trajectory, for example i put one FP at 50 where the trajectory length is 200,
i expected the output in this case will be 0.5 for JSC, since we have one FP and one TP(which is the length of trajectory). but in the leaderboard, the JSC score drops to 0 even if i added a one FP/FN.
I'd like to know whether this is intended or not, like if there is a single FP/FN in the single-state trajectory scenario, the entire score drops to 0. (so the single state scenario only has JSC 0 or 1)
If so, it seems the length of the trajectory isn't consdiered as TP for the evaluation..
Best
Junwoo
Hi Junwoo,
indeed, the length of the trajectory is not considered for the TP evaluation, sorry that my previous message was misleading on that (I have deleted the message to prevent misleading other readers). Before entering the CP detection metric, the last point (i.e. the length) is popped from the prediction. I have revised the code and indeed we had an unwanted behavior with ground truths (GT) with no changepoints. We have changed it such that, if the GT had no changepoint, and you correctly predict no CP, then this counts as a true positive (TP). If you instead predict one CP, it will count as one false positive (FP). Note that the JSC is calculated per experiment, in which we collect all FP, TP, FN and then calculate the JSC. Let me give an example, considering that the program already popped the T for each trajectory:
GT = [ [ ], [ ] ] # both GT trajectories have no CP
pred = [ [123], [ ] ] # one of the pred has a CP, the other one None
The first pred will account for one FP, the second will account for one TP (correctly predicted no CP), which means that the JSC = 1 / 2. Hope this solves the issue. Thanks for your collaboration, this helps a lot!
Gorka
Posted by: gorka.munoz @ May 7, 2024, 6:49 a.m.