SHARED TASK @

CONSTRAINT 2021

First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Si​tuation

Collocated with AAAI 2021

8th February, 2021


NEWS

ABOUT THE SHARED TASK

* Please use the following bibtex to cite the overview paper.


@inproceedings{patwa2021overview,
title={Overview of CONSTRAINT 2021 Shared Tasks: Detecting English COVID-19 Fake News and Hindi Hostile Posts },
author={Parth Patwa and Mohit Bhardwaj and Vineeth Guptha and Gitanjali Kumari and Shivam Sharma and Srinivas PYKL and Amitava Das and Asif Ekbal and Shad Akhtar and Tanmoy Chakraborty},
booktitle = {Proceedings of the First Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation ({CONSTRAINT})},
year = {2021},
publisher = {Springer},
}

* Please use the following bibtex to cite the english data paper.

@misc{patwa2020fighting,
      title={Fighting an Infodemic: COVID-19 Fake News Dataset}, 
      author={Parth Patwa and Shivam Sharma and Srinivas PYKL and Vineeth Guptha and Gitanjali Kumari and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
      year={2020},
      eprint={2011.03327},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

* Please use the following bibtex to cite the hindi data paper.

@misc{bhardwaj2020hostility,
      title={Hostility Detection Dataset in Hindi}, 
      author={Mohit Bhardwaj and Md Shad Akhtar and Asif Ekbal and Amitava Das and Tanmoy Chakraborty},
      year={2020},
      eprint={2011.03588},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

  • Tasks- The CONSTRAINT-2021 shared task on the hostile post detection invites participation in two subtasks:

    • COVID19 Fake News Detection in English - This subtask focuses on the detection of COVID19-related fake news in English. The sources of data are various social-media platforms such as Twitter, Facebook, Instagram, etc. Given a social media post, the objective of the shared task is to classify it into either fake or real news. For example, the following two posts belong to fake and real categories, respectively.

      If you take Crocin thrice a day you are safe. Fake
      Wearing mask can protect you from the virus Real

      English Dataset: https://competitions.codalab.org/competitions/26655

      English dataset paper: Fighting an Infodemic: COVID-19 Fake News Dataset

    • Hostile Post Detection in Hindi -This subtask focuses on a variety of hostile posts in Hindi Devanagari script collected from Twitter and Facebook. The set of valid categories are fake news, hate speech, offensive, defamation, and non-hostile posts. It is a multi-label multi-class classification problem where each post can belong to one or more of these hostile classes. However, the non-hostile posts cannot be grouped with any other class. The evaluation of this subtask will be two-dimensional as follows:
      1. Coarse-grained evaluation: It is a binary evaluation of hostile vs non-hostile posts.
      2. Fine-grained evaluation: It is a fine-grained evaluation of the hostile classes.

      Definitions of the class labels:
      • Fake News: A claim or information that is verified to be not true.
      • Hate Speech: A post targeting a specific group of people based on their ethnicity, religious beliefs, geographical belonging, race, etc., with malicious intentions of spreading hate or encouraging violence.
      • Offensive: A post containing profanity, impolite, rude, or vulgar language to insult a targeted individual or group.
      • Defamation: A mis-information regarding an individual or group.
      • Non-hostile: A post without any hostility.

      Examples

      ये देखो इस्लाम क्या क्या सिखाता है जिहाद से लेकर आतंकवादी और दंगों से लेकर चोरी बुर्खे की आड़ में चद्दर चुराती महिलाएं {hate}
      मोहतरमा JNU की 43 साल की छात्रा हैं , और कमाल की बात है कि उनकी बेटी मोना भी JNU में ही पड़ती है {Fake, Defamation}
      जब इन दलितों को (सभी नहीं) हिन्दू धर्म और हिन्दू देवी देवताओं से इतनी नफरत भारी हुई है तो धूर्त कहीं के अपना नाम हिन्दुओं के जैसे ही क्यों रखते हैं। किसने रोका है कुछ भी बन से, बन जाओ मुस्लिम, ईसाई और जो मन करे। इस धूर्त की हिम्मत नहीं कि किसी दूसरे धर्म के बारे ऐसा बोल दे । {Hate, Offensive}
      डॉक्टर कफ़ील ख़ान को हाईकोर्ट से मिली ज़मानत https://t.co/DH5WE370XT {Non-hostile}

    • Evaluation Metric: The official evaluation metric for the shared task is weighted-average F1 score.
      Hindi Dataset: https://competitions.codalab.org/competitions/26654
      Hindi Dataset Paper: Hostility Detection Dataset in Hindi
      Submission: Each team should submit a csv file in the following format for the final evaluation:
      <unique_id, {labels}>
      In case of multiple submissions by a team, we shall consider the last submission prior to the deadline for the final evaluation. No exceptions shall be made.
      System description paper: All team/participants will be invited to submit their models as short papers to be included in the proceedings. Based on the reviewers' comments, we will decide which papers to be accepted. There will be 2 types of accepted papers:
      1) Archival - These will be published in springer. At least one author must register for the workshop.
      2) (optional) Non - archival - These will not be published in formal prceedings. They will be (optional) presented as posters and we will put up a (optional) link to these papers on our website.
      Submission details: All papers must be submitted via our EasyChair submission page. Papers should be single blind. Minimum 6 pages and maximum 12 pages including references. paper template : https://www.springer.com/gp/authors-editors/conference-proceedings/conference-proceedings-guidelines
      Best paper and Task Winner Awards: There are 4 awards. Each one also includes a discounted registration for AAAI.
      • Winner of English shared task
      • Winner of Hindi shared task (fine grained)
      • Best paper award (main track)
      • Best paper award shared task (based on analysis, writing, methodology)

SHARED TASK

  • Important Dates:
    • October 1, 2020: Release of the training set
    • December 1, 2020: Release of the test set
    • December 10, 2020: Deadline for submitting the final results
    • December 12, 2020: Announcement of the results
    • December 25, 2020: System paper submission deadline (All teams are invited to submit a paper)
    • January 8 Notification of system description paper
    • January 14 Camera ready
    • February 8, 2020: CONSTRAINT workshop at AAAI

CONTACT US

Email
  tanmoy@iiitd.ac.in
  parthprasad.p17@iiits.in

Subscribe to our mailing list      Click Here

Follow Us