I agree with you. I believe contact should have been made before sealing the leaderboard.
In my own teams case, we dropped from 5th to 18th position. Apparently our solution didn't magically run on their side, meanwhile all is well on our side. Contacting us before sealing would have been a good idea. It's quite a thing that code that runs on one's environment may not run on some other. Then again, we had 64 submissions at the end of the competition, I don't understand how it's currently only 12.
Communication would really be key in such circumstances.
Agree with you brother on this. The matter on our side is resolved, we will be moved back to the 2nd place. Yes the environment will always be somewhat different. Also the package rule of always running the latest version is bit absurd. In competitions that run 2-3 months long or even more, many packages will change their versions. Maybe we have run our code with version 11 and when they check it's version 12. So again it will cause issues of reproducibility.
There is a possibility of score difference due to different versions being released.Just like if you used catboost,you're likely to have a different score on private lb since a newer version was released on 26th May.I think zindi should consider that
'%3e%3ccircle r='20' fill='%23008'/%3e%3ccircle r='17.5' fill='%23fff'/%3e%3ccircle r='3.5' fill='%23008'/%3e%3cg id='d'%3e%3cg id='c'%3e%3cg id='b'%3e%3cg id='a' fill='%23008'%3e%3ccircle r='.875' transform='rotate(7.5 -8.75 133.5)'/%3e%3cpath d='M0 17.5.6 7 0 2l-.6 5L0 17.5z'/%3e%3c/g%3e%3cuse xlink:href='%23a' transform='rotate(15)'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='rotate(30)'/%3e%3c/g%3e%3cuse xlink:href='%23c' transform='rotate(60)'/%3e%3c/g%3e%3cuse xlink:href='%23d' transform='rotate(120)'/%3e%3cuse xlink:href='%23d' transform='rotate(-120)'/%3e%3c/g%3e%3c/svg%3e)
I agree with you. I believe contact should have been made before sealing the leaderboard.
In my own teams case, we dropped from 5th to 18th position. Apparently our solution didn't magically run on their side, meanwhile all is well on our side. Contacting us before sealing would have been a good idea. It's quite a thing that code that runs on one's environment may not run on some other. Then again, we had 64 submissions at the end of the competition, I don't understand how it's currently only 12.
Communication would really be key in such circumstances.
Agree with you brother on this. The matter on our side is resolved, we will be moved back to the 2nd place. Yes the environment will always be somewhat different. Also the package rule of always running the latest version is bit absurd. In competitions that run 2-3 months long or even more, many packages will change their versions. Maybe we have run our code with version 11 and when they check it's version 12. So again it will cause issues of reproducibility.
There is a possibility of score difference due to different versions being released.Just like if you used catboost,you're likely to have a different score on private lb since a newer version was released on 26th May.I think zindi should consider that