Primary competition visual

GeoAI Amazon Basin Secret Runway Detection Challenge by ITU

Helping Austria, Peru
and 1 other country
  • Austria
  • Peru
  • Italy
  • Scroll to see more
2 000 CHF
Completed (almost 2 years ago)
Classification
Earth Observation
398 joined
26 active
Starti
09 Aug 24
Closei
11 Nov 24
Reveali
11 Nov 24
User avatar
meganomalyZindi Staff
Zindi staff member
Zindi
Sample Submission & Scoring
Platform · 12 Sep 2024, 13:51 · 7

Hello Participants!

We hope you’re ready and excited to tackle the GeoAI Amazon Basin Secret Runway Detection Challenge with ITU and Zindi.

For this challenge the error metric is Accuracy. However, the scoring will work slightly differently to our normal challenges.

For every predicted runway you will create a buffer region of 20 pixels around the runway. This buffer should extend 20 pixels in all directions ((left, right, top, and bottom). By adding this buffer you will submit a reduced submission file of max 200 000 rows. !NB Note the sample submission file shared is >25m rows - you will not submit this but rather your reduced submission file.

The reference file of ground truth labels against which you will be scored has also been reduced to ~ 200 000 rows. In the scoring process your file will be merged with the reference file. You will be scored on matching IDs. When the competition closes selected submissions from the top 10 participating users/teams will be scored fully.

Happy coding!

Discussion 7 answers
User avatar
marching_learning

Great !!!

Upvotes 0
User avatar

Should the reduced submission file include only the rows with detected pixels (label being equal to 1)?

Upvotes 0
User avatar
meganomalyZindi Staff
Zindi staff member

Your submission file should contain 2 columns:

tile_row_column, label

tile_row_column has tile name and label would be 0 or 1 (1 for detected runways)

Your rows will be determined by your detected runways and your buffer region around those. You should be submitting ~200k rows.

Upvotes 0
User avatar

Re:

"For every predicted runway you will create a buffer region of 20 pixels around the runway. This buffer should extend 20 pixels in all directions ((left, right, top, and bottom). By adding this buffer you will submit a reduced submission file of max 200 000 rows. !NB Note the sample submission file shared is >25m rows - you will not submit this but rather your reduced submission file."

Could you elaborate a bit on what you mean by adding a buffer and how this leads to a reduced submission file?

Upvotes 4
User avatar

Totally agree that wasn't clear. e.g. what if there's a tile posititive on the edge. does the buffer only extend inward? need to get more info on this "buffer"...

Upvotes 1
User avatar

Regarding the buffer region, as I understand it, you've incorporated this on the ground truth data that you will use to judge entries. Meaning that you draw a line on the runway and then all the indexes that are within the buffer should have value 1. I've assumed that the indexes in the submission file correspond to pixels.

I've also added a buffer when preparing my training data, but I've found there are many ways to do it. One can make a circle with radius 20 on every pixel on the runway and mark all that with one. One can also make a cross on every pixel, marking the 20 pixels on every side of every pixel with 1. This is taking @meganomaley 's instructions more literally.

This may seem like a nitpick, but by plotting these alternatives, I've seen that it changes a lot about how the buffer area ends up looking. (more specifically, it might change by a factor sqrt(2))

If one of the organizers (eg, @Lolletti @meganomaley) could have a look, I'd really appreciate it!

Upvotes 0
User avatar

Sorry, I misspelled you, @meganomaly

Here is a picture of a cross-shaped buffer, so it's clearer what I mean: https://imgur.com/a/iFxtTlM

Upvotes 0