My understanding from conversations in Zindi and Discord is we are free to use any teacher model, synthetic data, etc that we wish with the main caveats being:
- Results must be reproducible
- Only one model (not separate models for each task)
- InkubaLM must be used as the model your work is based on. So you would have to use InkubaLM or a pruned version of InkubaLM as your student model.
'%3e%3ccircle r='20' fill='%23008'/%3e%3ccircle r='17.5' fill='%23fff'/%3e%3ccircle r='3.5' fill='%23008'/%3e%3cg id='d'%3e%3cg id='c'%3e%3cg id='b'%3e%3cg id='a' fill='%23008'%3e%3ccircle r='.875' transform='rotate(7.5 -8.75 133.5)'/%3e%3cpath d='M0 17.5.6 7 0 2l-.6 5L0 17.5z'/%3e%3c/g%3e%3cuse xlink:href='%23a' transform='rotate(15)'/%3e%3c/g%3e%3cuse xlink:href='%23b' transform='rotate(30)'/%3e%3c/g%3e%3cuse xlink:href='%23c' transform='rotate(60)'/%3e%3c/g%3e%3cuse xlink:href='%23d' transform='rotate(120)'/%3e%3cuse xlink:href='%23d' transform='rotate(-120)'/%3e%3c/g%3e%3c/svg%3e)
My understanding from conversations in Zindi and Discord is we are free to use any teacher model, synthetic data, etc that we wish with the main caveats being:
- Results must be reproducible
- Only one model (not separate models for each task)
- InkubaLM must be used as the model your work is based on. So you would have to use InkubaLM or a pruned version of InkubaLM as your student model.