Preprocessed image recognition

Organized by areal - Current server time: Oct. 27, 2020, 11:42 a.m. UTC

Previous

Development Phase
Nov. 15, 2018, midnight UTC

Current

Final Phase
April 30, 2019, midnight UTC

End

Competition Ends
Never

Solve an Aerial image recognition problem

The raw data set contains 9100 color images of size 128 x 128, images of 13 class, each having 700 images. This challenge was created with preprocessed images, being vectors of 1024 features. This kind of data no longer looks like images, and we have no idea what those features can represent.
The classes are some kind of landscape or a natural environment, and more specifically :
beach, chaparral, cloud, desert, forest, island, lake, meadow, mountain, river, sea_ice, snowberg, wetland.

The goal of the challenge will be to classify each example and assign it to its correct class.

This challenge is relatively easier than the one on the raw data. Indeed, it is possible to obtain good performance thanks to a wide choice of learning algorithm, whether with deep learning or a simpler algorithms.

Below are some sample feature vectors visualized as images (matrices):

The challenge video presentation is below :

References and credits:
Gong Cheng, Junwei Han, and Xiaoqiang Lu, RemoteSensing Image Scene Classification: Benchmark andState of the Art. IEEE International.
The competition protocol was designed by Isabelle Guyon.
The starting kit was adapted from an Jupyper notebook designed by Balazs Kegl for the RAMP platform.
This challenge was generated using Chalab, a competition wizard designed by Laurent Senta.
This challenge was created by the Areal team, composed of David Biard, Samuel Berrien, Théo Cornille, Robin Duraz, Hao Liu and Trung Vu-Thanh. You can contact the team at areal@chalearn.org.
The original data is a large-scale dataset, termed "NWPU-RESISC45", which is a publicly available benchmark for Remote Sensing Image Scene Classification (RESISC), created by Northwestern Polytechnical University (NWPU). This dataset contains 31,500 images, covering 45 scene classes with 700 images (256 x 256 pixels) per class.

Evaluation

The problem is a multicalss classification problem. Each sample (an image) is characterized by its 4096 features. You must predict the categories of 13 classes.
You are given for training a data matrix X_train of dimension 5200 samples x 1024 features and an array y_train of labels of dimension 5200 samples. You must train a model which predicts the labels for two test matrices X_valid and X_test, each having 1950 samples.
There are 2 phases:

  • Phase 1: development phase. We provide you with labeled training data and unlabeled validation and test data. Make predictions for both datasets. However, you will receive feed-back on your performance on the validation set only. The performance of your LAST submission will be displayed on the leaderboard.
  • Phase 2: final phase. You do not need to do anything. Your last submission of phase 1 will be automatically forwarded. Your performance on the test set will appear on the leaderboard when the organizers finish checking the submissions.

This sample competition allows you to submit either:

  • Only prediction results (no code).
  • A prediction model that must be trained and tested.

The submissions are evaluated using the Accuracy metric. This metric determines de classification quality and it is computed by dividing the number of true positive (data correctly classified) by total number of processed data. This kind of metric is at the same time simple and informative on the performance for this classification task given that the class distribution within this project is balanced.

Rules

Submissions must be made before the end of phase 1. You may submit 5 submissions every day and 100 in total.

This challenge is governed by the general ChaLearn contest rules.

Development Phase

Start: Nov. 15, 2018, midnight

Description: Development phase: tune your models and submit prediction results, trained model, or untrained model.

Final Phase

Start: April 30, 2019, midnight

Description: Final phase (no submission, your last submission from the previous phase is automatically forwarded).

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 reference2 0.8969
2 orbiter 0.8944
3 grp.Sputnik 0.8903